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Abstract. Using the norm of the Higgs field as a Morse function, we 
study the moduh spaces of U(p, r/)-Higgs bundles over a Riemann surface. We 
require that the genus of the surface be at least two, but place no constraints 
on {p,q). A key step is the identification of the function's local minima as 
moduli spaces of holomorphic triples. We prove that these moduli spaces of 
triples are irreducible and non-empty. 

Because of the relation between fiat bundles and fundamental group rep- 
resentations, we can interpret our conclusions as results about the num- 
ber of connected components in the moduli space of semisimplc PU(j9, q)- 
representations. The topological invariants of the fiat bundles bundle are 
used to label components. These invariants are bounded by a Milnor-Wood 
type inequality. For each allowed value of the invariants satisfying a certain 
coprimality condition, we prove that the corresponding component is non- 
empty and connected. If the coprimality condition does not hold, our results 
apply to the irreducible representations. 
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1 Introduction 



The core of this paper is a Morse theoretic study of the the moduh space 
of U(p, g)-Higgs bundles over a Riemann surface X of genus g ^ 2. Our 
interest in this space comes from two sources. The first is its relevance to 
questions concerning the representation variety for representations of tciX 
in the real Lie group PU(p, g). The second has to do with the intrinsic 
geometry revealed by the Morse function and the methods we are able to use 
to carry out our analysis. Our main goals are to fully understand the minimal 
submanifolds of the Morse function and, thereby, to count the number of 
connected components in the representation varieties. 

A Higgs bundle consists of a holomorphic bundle together with a Higgs 
field, i.e. a section of a certain associated vector bundle. A U{p, g)-Higgs 
bundle is a special case of the G-Higgs bundles defined by Hitchin in pB| , 



where G is a real form of a complex reductive Lie group. Such objects provide 
a natural generalization of holomorphic vector bundles, which correspond to 
the case G = U{n) and zero Higgs field. In particular, they permit an 



extension to other groups of the Narasimhan and Seshadri theorem (|p8||) 
on the relation between unitary representations of tciX and stable vector 
bundles. 

By embedding U(p, q) in GL{p + q) we can give a concrete description of 
a U(p, g)-Higgs bundle as a pair 

where V and W are holomorphic vector bundles of rank p and q respectively, 
/3 is a section in if°(Hom(W", V') (g) K), and 7 G H^(Rom{V,W) O K), so 
that $ G H°{End{V ®W)(g)K). Foremost among the key features of such 
objects is (by the work of Hitchin, [^S], Q Donaldson |12[, Corlette [|10| and 



Simpson [0, ^ |3^, the existence of moduli spaces of polystable objects 
which can be identified with moduli spaces of solutions to natural gauge 
theoretic equations. Moreover, since the gauge theory equations amount 
to a projective flatness condition, these moduli spaces correspond with a 
moduli spaces of fiat structures. In the case of U(p, g)-Higgs bundles, the fiat 
structures correspond to semisimple representations of tiiX into the group 
PU(p, q). The Higgs bundle moduli spaces can thus be used, in a way which 
we make precise in Sections ^ and ^, as a tool to study the representation 
variety 

7^(PU(p,g)) = Hom+(7riX,PU(p,g))/PU(p,g) , 

where Hom^(7riX, PU(p, q) ) denotes the semisimple representations into PU(p, q) 
and the the quotient is by the adjoint action. 
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This relation between Higgs bundles and surface group representations 
has been successfully exploited by others, going back originally to the work 
of Hitchin and Simpson on complex reductive groups. The use of Higgs bun- 
dle methods to study representation variety 7^(G) for real G was pioneered 
by Hitchin in pil, and further developed in [jl8|, |l9i and by Xia and Xia- 



Markman in ^ ^ Where we differ from these works is that in 
none of them is the general case of PU(p, q) considered. What we have in 
common is that in all cases insight into the topology of the space 'R-{G) comes 
from a natural Morse function on the corresponding moduli space of Higgs 
bundles. 

The natural Morse function measures the L^-norm of the Higgs field. 
This turns out to provide a suitably non-degenerate Morse function which 
is, moreover, a proper map. In some cases (cf. |2^, ^) all the critical sub- 
manifolds are sufficiently well understood so as to permit the extraction of 
detailed topological information such as the Poincare polynomial. In our case 
our understanding is confined mostly to the submanifolds corresponding to 
the local minima of the Morse function. Fortunately, this is sufficient for 
our purposes, namely to understand the number of components of the Higgs 
moduh spaces, and thus of the representation varieties. 

The Morse function is non-negative but cannot always attain its zero 
lower bound. For GL(n)-Higgs bundles, this lower bound is attained, with 
the minimizing points in the moduli space consisting of semistable vector 
bundles with zero Higgs field. However in the case of U(p, g)-Higgs bundles, 
the special form of the underlying holomorphic bundle prevents a polystable 
Higgs bundle from having a vanishing Higgs field. The minimizers of the 
Morse function thus have a more complicated structure than simply that 
of a stable bundle. Generalizing the results in |T8|, |19[, we show that any 
minimizer consists of a pair of bundles together with a morphism between 
them. That is, the minimizers correspond precisely to a special case of the 
holomorphic triples introduced in 0. 

The holomorphic triples admit moduli spaces of stable objects in their 
own right. In order to exploit the relation between these spaces and the 
minimal submanifolds in the moduli spaces of U(p, g)-Higgs bundles, we need 
a sufficiently good understanding of the triples moduli spaces. A substantial 
part of this paper is devoted to acquiring just such an understanding. The 
way we acquire the needed information is similar in spirit to techniques used 
by Thaddeus in |3^. The key idea (described fully in sections |^ and ||) is 
that the moduli spaces of triples come in discrete families, with the members 
of the families ordered by intervals in the range of a continuously varying 
real parameter. As the parameter moves to the large extreme of its range. 



2 



the structure of the corresponding moduh spaces simphfy and we can obtain 
a detailed description. Moreover, as the parameter decreases, we can track, 
albeit somewhat crudely, how the moduli spaces change. Combining these 
pieces of data, we get just enough information about the moduli space of 
relevance to our Higgs bundle problem. 

We now give a brief summary of the contents and main results of this 
paper. 

In Sections ^ and |^ we give some background and describe the basic ob- 
jects of our study. In Sections we describe the natural invariants associated 
with representations of ttiX into PU(p, q). We also discuss the invariants as- 
sociated with representations of F, the universal central extensions of tti, 
into \]{p,q). In both cases, these involve a pair of integers (a, 6) which can 
be interpreted respectively as degrees of rank p and rank q vector bundles 
over X. In the case of the PU(p, q) representations, the pair is well defined 
only as a class in a quotient Z© Z/ (p, g)Z. This leads us to define subspaces 
TZ[a, b] C 7^(PU(p, q)) and 7^r(a, b) C 7^r(U(p, <?))• For fixed (a, b), the space 
7^r(a, fibers over 7l[a,b] with connected fibers. 

In section ^ we give precise definitions of the U(p, g)-IIiggs bundles and 
their moduli spaces and establish their essential properties. Thinking of 
a U(p, g) -Higgs bundle as a pair (V © W,^), the parameters (a, 6) appear 
here as the degrees of the bundles V and W. We denote the moduli space of 
polystable U(p, g)-Higgs bundles with deg(V") = a and deg W = bhj Ai{a,b), 
and identify A4{a,b) with the component 7?.r(a, 6) of 7lr(JJ{p,q))- This, 
together with the fibration over 7^r(U(p, q)) are the crucial links between the 
Higgs moduli and the surface group representation varieties. 

Except for the last section, where we translate back to the language of 
representation varieties, the rest of the paper is concerned with the spaces 
J\4{a,b). Fixing p, q, a and b, we begin the Morse theoretic analysis of Ai (a, b) 
in Section ^. Using the L^-norm of the Higgs field $ = ( ° q ) as the Morse 
function, the basic result we need (cf. Proposition [4.2| ) is that this function 
has a minimum on each connected component of Ai{a, b), and if the subspace 
of local minima is connected then so is A^(a, b). The next step is to identify 
the local minima, the loci of which we denote by A/'(a, b) . We prove (cf. 
Propositions |4.10| and |4.15| ) that these correspond precisely to the polystable 
Higgs bundles in which /3 = or 7 = 0. The data defining a Higgs bundle 
with (3 = can thus be written as the triple {W © K, V,7). Similarly, the 
7 = minima correspond to triples {V © K,W,(3). This brings us to the 
theory of such holomorphic triples. 

In sections we develop the theory we need concerning holomorphic 
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triples and their moduli spaces. While only triples of a specific special kind 
correspond to the minima on the U(p, g)-Higgs moduli, we develop the theory 
for the general case in which a holomorphic triple is specified by the set T = 
(£"1, E2,4>), where Ei and E2 are holomorphic bundles on X and (p: E2 —>■ Ei 
is holomorphic (see |15] and |^). There is a notion of stability for triples 



which depends on a real parameter a and there are moduli spaces of a- 
polystable triples, which are shown in P] (see also flSl) to be projective 
varieties. In order for A/'^ to be non-empty, one must have a ^ with 
<^m = di/rii — d2/n2 ^ 0. In the case rii 7^ n2 there is also a finite upper 
bound aM- When the parameter a varies, the nature of the a-stability 
condition only changes for a discrete number of so-called critical values of a 
(see section |0| for the precise statements). We denote by 

J^a = Maini,n2,di,d2) 

the moduli space of a-polystable triples with rk(i?j) = rij and deg(£'i) = di 
for i = 1, 2. The subspace of a-stable triples inside Ma, denoted by Af^, is a 
quasi-projective variety. In Theorem f).21\ we show that 

Theorem. Af^ is smooth for all values of a greater than or equal to 2g — 2. 

We show furthermore that the triples which appear in J\f{a, h) are a-polystable 
with a = 2g — 2. We must thus understand a moduli space of a-stable triples, 
with a on the boundary of the range in which the moduli spaces are smooth. 
We do this indirectly, by obtaining a description of Af^ when a is large and 
then examining how the moduli space changes as a decreases. 

In section ^ we examine how the moduli spaces differ for values of a on 
opposite sides of a critical value. If J\f^± denote the moduli spaces for values 
of a above and below a critical value ac, we denote the loci along which they 
differ by S^± respectively. Our main results are 

Theorem (Theorem |6.19| ). Let ac € (a^, c^m) be a critical value for triples 
of type {ni,n2, di, ^2)- If etc > 2g — 2 then the loci S^± C Af'^± are contained 
in subvarieties of codimension at least g — 1. In particular, they are contained 
in subvarieties of strictly positive codimension if g ^ 2. If ac = 2g — 2 then 
the same is true for . 



Theorem (Corollary |6.20|) . Let ai and a2 be any two values in {amiOiM) 



such that am < ai < 02 < oim and 2g — 2 ^ ai. Then 

• The moduli spaces A/"^^ and Af^^ have the same number of connected 
components, and 
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• The moduli space Af^^ is irreducible if and only if Af^^ is. 

Where by aM we denote the upper bound for a if ni 7^ n2, or cx) if 
rii = n2. 

In sections we look more closely at how the parameter a affects the 
nature of a-stable triples. There are three cases to consider, namely ni < n2, 
rii > n2 and ni = n2. However, using a duality result, it is enough to 
consider rii > n2 and rii = n2. In the first C3iSG, clS mentioned above, there 
is a bounded interval [am,aAf] outside of which Ma is empty. Within this 
interval we identify a number of special values beyond which the structure of 
a-stable triples simplify; by Corollary [7.3| the map ip : E2 ^ Ei is injective 
if a > ao, by Proposition |7.5| the cokernel is torsion free. Finally, for the 
largest values of a, i.e. for values greater than a bound which we denote by 
ai, we show (cf. Proposition |8.3| ) that a-stable triples have the form 

— > E2^ El — ^ F — ^ 0, 

with F locally free, and E2 and F semistable. This leads to a description of 
Afa for any a in the range < a < aM- Denoting this moduli space by A/l, 
we get 

Theorem (Theorem |8.7| ). Let ni > n2 and d\jn\ > d2/n2. 

The moduli space N'[{ni,n2,di,d2) is smooth, and is birationally equiva- 
lent to a -fibration over M^{ni — n2, di — ^2) x M*(n2, d2), where M''^{n, d) 
denotes the moduli space of stable bundles of degree n and rank d, and the 
fiber dimension is N = n2di — nid2 + ni{ni — n2){g — 1) — 1. In particular, 
J\fl{ni,n2,di,d2) is non-empty and irreducible. // GCD(?2i — ^2, c/i — ^2) = 1 
and GCD(n2,(i2) = 1, the birational equivalence is an isomorphism. 

Moreover, A/L(ni, n2, c/i, ^2) is irreducible and hence birationally equiva- 
lent to Af[{ni,n2, di, (^2). 

Theorem (Theorem |8.9| , Corollary |8.10| ). Let a be any value in the range 
2g — 2 ^ a < aM- Then Af^ is birationally equivalent to Afl. In particular it 
is non-empty and irreducible. 

Let (ni, ^2, (ii, (^2) be such that GCD(n2,ni +n2,di + ^2) = 1- If a is 
generic then Afa is birationally equivalent to A/l, and in particular it is irre- 
ducible. 

The case rii = 712 differs from the ni > 712 case in two ways. The range 
for a is unbounded above, and in general there is no way to avoid torsion 
in the cokernel of the map (p. The range for a presents no difficulties since 
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(cf. Theorem |9.5| ) beyond a finite bound there are no changes in the moduh 
spaces. It thus still makes sense to identify a 'large a moduli space, Ml ■ We 
prove the following. 



Theorem (Theorem |9.13|) . The moduli space Miin, n, di, ^2) is non-empty 
and irreducible. Moreover, it is birationally equivalent to a -fibration over 
M'^{n, ^2) X Div'^(X), where the fiber dimension is N = n{di — ^2) — 1- 



Theorem (Theorem |9.14|) . If a ^ 2(7—2 then the moduli spaceM^ln, n, di, 6/2) 
is birationally equivalent to J\fL{n,n,di,d2) and hence non-empty and irre- 
ducible. Moreover, J\fa{n, n, di, c/2) is birationally equivalent to A/L(n, n, di, c?2), 
and hence irreducible, if also 

• GCD(r;,, 2n, di + ^2) = 1 and a ^ 2g — 2 is generic, or 

• di ~ d2 < a. 

In section we apply our results to the moduli spaces A^(a,6), and 
hence to the components TZr{a,b) and TZ[a,b] of the representation varieties 
7?.r(U(p, g)) and 7?.(PU(p, g)), respectively. Some of the results depend only 
on the combination 

aq — bp 



T{a, b) 



p + q 



known as the Toledo invariant. Indeed, (a, b) is constrained by the bounds 
^ |r| ^ tm, where tm = 2mm{p,q}{g — 1). Originally proved by Domic 



and Toledo in , these bounds emerge naturally from our point of view (cf . 
Corollary 3.21 and Remark 5.1^j ). After a discussion (in Section 10.1 ) of the 



relation between (a, b) and r, and (in section |10.2| ) of the significance of the 
coprime condition GCD(p + g, a + 6) = 1, we assemble (in section |10.3|) our 



results for the Higgs Moduli spaces. Summarizing the results of section [10.3 
into one Theorem, we get 

Theorem. Let {a,b) be such that |r(a, 6)| ^ tm. Unless further restrictions 
are imposed, let {p,q) be any pair of positive integers. 



(1) // either of the following sets of conditions apply, then the moduli space 
7W*(a, 6) is a non-empty, smooth manifold of the expected dimension, 
with connected closure M^{a,b): 

(i) < \T{a,b)\ < Tm , 
(ii) |r(a, b) \ = Tm and p = q 
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(2) // any one of the following sets of conditions apply, then the moduli 
space /iA{a,b) is non-empty and connected: 

(z) r(a,6) = 0, 

(m) |r(a, 6)1 = tm and p ^ q. , 

(Hi) {p - l){2g - 2) < |r| ^ tm = p{2g - 2) and p = q, 
{iv) GCD(p + g, a + 6) = 1 

(3) // |r(a,6)| = Tm and p ^ q then any element in A4{a,b) is strictly 
semistable (i.e. A1''(a,6) is empty). Ifp < q, then any such representa- 
tion decomposes as a direct sum of a l]{p,p)-Higgs bundle with maximal 
Toledo invariant and a polystable vector bundle of rank q — p. Thus, if 
T = p{2g — 2) then there is an isomorphism 

M{p,q,a,b)^Mip,p,a,a-p{2g-2)) x M{q ~ p,b - a + p{2g ~ 2)), 

where the notation A4{p,q,a,b) indicates the moduli space ofl]{p,q)- 
Higgs bundles with invariants {a,b), and M{n,d) is the moduli space 
of semistable vector bundles of rank n and degree d. (A similar result 
holds if p > q, as well as if t = —p{2g — 2) ). 

(4) // GCD(p + g, a + 6) = 1 then A4{a,b) is a smooth manifold of the 
expected dimension. 

Translating this into the language of representations of F and the funda- 
mental group, we get the following. 

Theorem (Theorem |10.18D . Let {a,b) be such that |r(a, 6)| ^ tm. Unless 
further restrictions are imposed, let {p,q) be any pair of positive integers. 

(1) // either of the following sets of conditions apply, then the moduli 
space 7?.p(a,6) of irreducible semi-simple representations, is a non- 
empty, smooth manifold of the expected dimension, with connected clo- 
sure TZ^i^a, b): 

{i) < |r(a,6)| < tm , 
{ii) \T{a,b)\ = Tm and p = q 

(2) // any one of the following sets of conditions apply, then the moduli 
space TZriO', b) of all semi-simple representations is non-empty and con- 
nected: 
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(z) r{a,b) = 0, 

(ii) \T{a,b)\ = tm and p ^ q. , 

{in) {p - l){2g - 2) < |r| ^ tm = p{2g - 2) and p = q, 

iiv) GCD(p + g, a + 6) = 1 

(3) // |r(a, 6)1 = tm and p ^ q then any representation in 7lr{a,b) is 
reducible (i.e. 7l^{a,b) is empty). If p < q, then any such representa- 
tion decomposes as a direct sum of a semisimple representation ofV in 
U(p,p) with maximal Toledo invariant and a semisimple representation 
in U(g — p). Thus, if r = p(2g — 2) then there is an isomorphism 

'JZr{p,q,a,b)='JZr{p,p,a,a-p{2g-2)) x R^(q - p,b - a + pi2g - 2)), 

where the notation 7lr{p, q, a, b) indicates the moduli space of represen- 
tations ofV in U(p, g) with invariants {a,b), and Rr{n,d) denotes the 
moduli space of degree d representations ofV in U(n). (A similar result 
holds if p > q, as well as if t = —p{2g — 2) ). 

(4) // GCD(]9 + g, a + 6) = 1 then 7?.r(a,6) is a smooth manifold of the 
expected dimension. 



Theorem (Theorem |10.19|) . Let {a,b) be such that |r(a,6)| ^ tm. Unless 



further restrictions are imposed, let {p,q) be any pair of positive integers. 



(1) // either of the following sets of conditions apply, then the moduli space 
7l*[a,b] of irreducible semi-simple representations, is non-empty, with 
connected closure TZ* [a, b] : 

{i) < |r(a,6)| < Tm , 
{ii) |r(a, b)\ = Tm and p = q 

(2) // any one of the following sets of conditions apply, then the moduli 
space TZ[a,b] of all semi-simple representations is non-empty and con- 
nected: 

(i) r(a,6) = 0, 

{ii) \T{a, b)\ = Tm and p q. , 

(Hi) {p - \)(2g - 2) < |r| ^ tm = p{2g - 2) and p = q, 

{iv) GCD(p + g, a + 6) = 1 
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(3) // |r(a,6)| = tm and p ^ q then any representation in TZ[a^h] is re- 
ducible (i.e. TZ*[a,h] is empty). If p < q, then any such representation 
reduces to a semisimple representation o/tciX in P{U{p,p) x U(g — p)), 
such that the representation in PU{p,p) induced via projection on the 
first factor has maximal Toledo invariant. (A similar result holds if 
p > q, as well as if t = —p{2g — 2) ). 



Statement (3) in the previous theorem is a generahzation to arbitrary 



{p, q) of a result of D. Toledo when p = 1 and L. Hernandez when 
p = 2. This rigidity phenomenon for the moduli space of representations 
for the largest value of the Toledo invariant turns out to be of significance 
in relation to Hitchin's Teichmiiller components for the real split form of a 
complex group ||2^ (this will be discussed somewhere else |1T6|). 

We note, finally, that our methods clearly have wider applicability than 
to the U(p, g)-Higgs bundles and representations into PU(p, g). A careful 
scrutiny of the Lie algebra properties used in our proofs suggests a general- 
ization to any real group G for which G/K is hermitian symmetric, where 
K (Z G is a. maximal compact subgroup. This will be addressed in a future 
publication. 

The main results proved in this paper were announced in the note |^ . In 
that note we claim that the connectedness results hold for the moduli spaces 
7?.(a, b) and 7l[a, b], whether or not the coprimality condition GCD(p + g, a + 
6) = 1 is satisfied (and similarly for the corresponding moduli of triples and 
U(p, g)-Higgs bundles). While we expect this to be true, we have not so far 
been able to prove it. We hope to come back to this question in a future 
publication. 
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2 Representations of surface groups 



2.1 Definitions 

Let X be a closed oriented surface of genus g ^ 2 and let G be either U(p, q) 
or PU(p, q) where p and q are any positive integers. We think of U(p, q) as the 
subgroup of GL(?2) (with n = p + q) which leaves invariant a hermitian form 
of signature (p, q). It is a non-compact real form of GL(n) with center and 
maximal compact subgroup U(p) x U(g). The quotient U(p, g)/U(p) x U(g) 
is a hermitian symmetric space. The adjoint form PU(p, q) is given by the 
exacts sequence of groups 

1 U(l) U(p, q) — . PU(p, g) 1. 

By a representation of ttiX in G we mean a homomorphism p: ttiX G. 
Fixing PU(p, g) C PGL(n), we say a representation of ttiX in PU(p, g) is 
semi-simple if it defines a semi-simple PGL(n) representation. The group 
PU(p, q) acts on the set of representations via conjugation. Restricting to 
the semi-simple representations, we get the character variety, 

7^(PU(p, q)) = Hom+(7riX, PU(p, g))/PU(p, q). (2.1) 

This can be described as follows: from the standard presentation 

9 

TTlX = (Al, 5l, . . . , Ag, Bg I fii] = 1) 

2 = 1 

we see that Hom"''(7riX, PU(p, g)) can be embedded in PU(p, g)^^ via 

Hom+(7riX, PU(p, g)) ^ PU(p, qf^ 

p^{p{A,),...p{Bg)). 

We give Hom^(7riX, PU(p, g)) the subspace topology and 7^(PU(p, g)) the 
quotient topology; this is Hausdorff because we have restricted attention to 
semi-simple representations. We can similarly define 

7^^(U(p, g)) = Hom+(r, U(p, g))/U(p, g), (2.2) 

where F is the central extension 

— >Z — ^ F — > TTiX — > 1 (2.3) 

defined (as in 0]) by the generators Ai.Bi,... ,Ag,Bg and a central element 
J subject to the relation nf=i[^i)-^«] — J- Regarding U(p, g) as a subset 
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of GL(n), the representations in Hom''"(r, U(p, g)) are direct sums of irre- 
ducible representations on C". The first step in the study of the topological 
properties of TZ{G) is to identify the appropriate topological invariants of a 
representation p: niX G. To do that, one uses the correspondence be- 
tween representations of ttiX in G and flat principal G-bundles on X. We 
start with G — PU(p, g). Let p: niX — > PU(p, g) be a representation. The 
corresponding flat principal PU(p, g')-bundle is 

Pp = X XpP\]{p,q), 

where X is the universal cover of X. Since X has real dimension two, any 
PU(p, g)-bundle lifts to a U{p, g)-bundle. Moreover, a PU(p, g)-bundle with 
a flat connection can be lifted to a U(]9, g)-bundle with a projectively flat 
connection, i.e. with a connection with constant central curvature. Now, 
for any smooth (not necessarily flat) U(p, g)-bundle there is a reduction of 
structure group to the maximal compact subgroup U(p) x U(g). Taking the 
standard representation on © C^, we get an associated vector bundle of 
the form © FT, where V and W are rank p and q complex vector bundles 
respectively. Such bundles over a Riemann surface are topologically classifled 
by a pair of integers 

(a,6) = (deg(y),deg(H/)). 

The lift to a U{p,q) -bundle, and therefore the pair {a,b), is however not 
uniquely determined. If we twist the associated vector bundle (plus projec- 
tively flat connection) by a line bundle L with a connection with constant 
curvature, then after projectivizing we obtain the same flat PU(p, g)-bundle. 
If the degree of L is Z then the invariant associated to the twisted bundle 
is (a + pl,b + ql). In order to obtain a well deflned invariant for the rep- 
resentation p we must thus take the quotient of (Z © Z) by the Z-action 
/ • (a, b) = {a + pl,b + ql), i.e. we must pass to the quotient (Z © Z)/(p, q)Z 
in the exact sequence 

O^Z^Z©Z^ (Z©Z)/(p,g)Z ^ . 

Since the PU(p, g)-orbits in Hom(7riX, PU(p, q)) under the conjugation ac- 
tion correspond to isomorphism classes of flat PU(p, g)-bundles, the above 
construction deflnes a map 

c : 7^(PU(p, q)) ^ (Z © Z)/(p, q). (2.4) 

The map is continuous and is thus constant on connected components of 
n{FV{p,q)). 
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Remark 2.1. This map can be seen from a different point of view, from which 
it seen that the target space is 7riPU(p, g). We begin with the observation 
that the flat bundle Pp is described by locally constant transition functions. 
Thus the isomorphism class of this bundle is represented by a class in the 
(non-abehan) cohomology set PU(p, g)), where, by abuse of notation, 

we denote the sheaf of locally constant maps into PU(p, q) on X by the same 
symbol PU(p, q). Let PU(p, q) be the universal cover of PU(p, q). The short 
exact sequence of groups 

TTiPUb, q) ^ PU(p, q) PU(p, q) 

induces a sequence of cohomology sets and, since 7riPU(p, q) is Abelian, the 
coboundary map 

5: H\X,V\]{p,q)) H\X,n^V\]{p,q)) 

can be defined. The obstruction to lifting the fiat PU(p, g)-bundle Pp to a 
fiat PU(p, g)-bundle is exactly the image of the cohomology class of Pp under 
5. We denote this class by c(p) G H'^{X,'KiP\]{p,q)) = 7riPU(p,g). Next 
we recall the calculation of 7riPU(p, g). The maximal compact subgroup of 
U(p, q) is U(p) X U(g) and the inclusion U(p) x U(g) =— > U(p, q) is a homotopy 
equivalence. The determinant gives an isomorphism of fundamental groups 
7riU(p) ^ 7riU(l) ^ Z. Hence the map U(p, g) ^ U(l) x U(l) defined by 

U(p,g)^U(l)xU(l) 

y\ (2.5) 

z w . 



(det(x), det(w)) 



gives an isomorphism 7riU(p, g) ^ Z © Z. Furthermore, the composition of 
the standard inclusion U(l) ^ U(p, g) and the map given in ( p.5|) is the map 
A 1-^ (A^, A'') from U(l) — U(l) x U(l). The induced map on fundamental 
groups is n I— > {pn,qn). The short exact sequence 

U(l)-^U(p,g)^PU(p,g) (2.6) 

is a fibration, so we see that 7ii\J{p, q) 7riPU(p, g) is surjective. It follows 
that we have a commutative diagram 

7riU(l) . vriU(p,g) . 7riPU(p,g) 



Z 



Z©Z 



(Z©Z)/(p,g)Z 



and hence c(p) defines a class [a, 6] G (Z © Z)/(p, g)Z. This is the same class 
as that defined by the map ( f^.4| ) though, since we will not make use of this, 
we omit the proof. 



12 



2.2 Invariants on 1Zy{V{p, q)) and relation to 7?.(PU(]?, q)) 



Putting together ( p.6|) and (|2.3| ) we get the commutative diagram 
U(l) > \J{p,q) > PU{p,q) 



From this we get a surjection vr : 7lr(JJ{p,q)) 7^(PU(p, g)). We can 
understand the fibers of this map as follows. By the same argument as in 
00) ^r(U(p, g)) can be identified as the moduh space of U(p, g)-bundles 
on X with projectively fiat structures. Taking the reduction to the maximal 
compact U(p) x U(g), we thus associate to each class p G 7^r(U(p, q)) a vector 
bundle of the form V © W, where V and W are rank p and q respectively, 
and thus a pair of integers (a, b) = (degiV), deg(W)). The map c : p {a,b) 
fits in a commutative diagram 



7^^(U(p,g)) 



z©z 



7^(PU(p,g)) 



(Z©Z)/(p,g)Z 



We can now define the subspaces 

TZr{o., h) : = c~^{a, h) 

= {pG7^^(U(p,g)) I c(p) = (a,6) gZ©Z}, 

= {p G 7^(PU(p, q)) I c(p) = [a, 6] G Z © Z/(p, g)Z} . 
Clearly we have surjective maps 

nr{a,b) n[a,b]. 

Moreover, the pre-image 



vr 



-\n[a,b])= [jnr{a,b) 

ia,b) 



(2.7) 



(2i 



where the union is over all (a, b) in the class [a, b] G Z © Z/ (p, g)Z. As men- 
tioned above, tensoring by line bundles with constant curvature connections 
of degree / gives an isomorphism 

7?.r(a, b) ^ 7^r(o + pl,b + ql) . 



^ While 1^ gives the argument for U(ri) and PU(n), there are no essential changes to 
be made in order to adapt for the case of U(p, q) and PU(p, q). 
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Notice that if the invariant c(p) of a representation p G 7?.(PU(p, g)) can 
be represented by the pair (a, —a), then the associated U(p, g)-bundle has 
degree zero and the projectively flat connection is actually flat. Thus p 
defines to a representation of ttiX in U(p, g). Under the correspondence 
between 7?.(PU(p, g)) and 7^r(U(P5 Q')), P corresponds to a F representation 
in which the central element J acts trivially. Furthermore the subspaces 

7l{a) = {p G lZ{}J{j>,q)) I c(p) = (a, —a) with a G Z} 

can be identified with the subspaces 7^r(a, — a) C 7?,r(U(p, g)). Finally, we 
observe that Jac(X), the moduli space of flat degree zero line bundles, acts 
by tensor product of bundles on 7?.r(a, h). Since Jac(X) is isomorphic to the 
torus U(l)^^, we get the following relation between connected components. 

Proposition 2.2. The map TZr{a,b) TZ[a,b] given in ( p.7|) defines a 
U {1)"^^ -fibration which, if the total space and base are smooth manifolds, is 
a smooth principal bundle. Thus the subspace 7l[a,b] C 7?.(PU(p, g)) is con- 
nected ifTZY{a,b) is connected. □ 

We will study 7^r(a., b) by choosing a complex structure on X and identi- 
fying this space with a certain moduli space of Higgs bundles. This is carried 
out in the next section. In the rest of the paper, the subspaces of irreducible 
representations are denoted by TZ*. 



3 Higgs bundles and flat connections 
3.1 GL(n)-Higgs bundles 

Give X the structure of a Riemann surface. We recall (from [^, 



33| , p4| ) the following basic facts about GL(n)-Higgs bundles. 



Definition 3.1. 1. A GL{n)-Higgs bundle on X is a pair {E, $), where E 
is a rankn holomorphic vector bundle over X and $ G if°(End(i?)(8>-ft') 
is a holomorphic endomorphism of E twisted by the canonical bundle 
K ofX. 

2. The GL{n)-Higgs bundle {E, $) is stable if the slope stability condition 

p{E') < p{E) (3.1) 

holds for all proper (^-invariant subbundles E' of E. Here the slope is 
defined by p{E) = deg{E)/ Tk{E) and ^-invariance means that $(i?') C 
E'^K. 
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3. Semistability is defined by replacing the above strict inequality with a 
weak inequality. A Higgs bundle is called polystable if is the direct sum 
of stable Higgs bundles with the same slope. 

4. Given a hermitian metric H on E, let A denote the unique connection 
compatible with the holomorphic structure and unitary with respect to 
H . Hitchin's equations on {E, $) are 

Fa + [$, $1 = -V^fxIdEU, 
= 0, 

where uo is the Kdhler form on X, Me is the identity on E, n = fi{E) 
and Oa is the antiholomorphic part of the covariant derivative dA ■ 

Proposition 3.2. ||, [||, |||, |^ 

1. Let be a GL{n)-Higgs bundle. Then (E,^) is polystable if and 
only if it admits a hermitian metric such that Hitchin's equation l\3.2j 
is satisfied 

2. There is a moduli space of rank n degree d polystable Higgs bundles 
which is a quasi-projective variety of complex dimension 2{d+n'^{g—l)) . 

3. If we define a Higgs connection ( as in / PS]/ ) by 

D = dA + 9 (3.3) 

where = $ + $*, then Hitchin's equations are equivalent to the con- 
ditions 

Fd = - \/^fiIdEUJ, 

dAO = 0, (3.4) 
d*Ae = 0. 

4. In particular, since X is a Riemann surface, if A satisfies (^.2J then 
D is a projectively flat connection. If deg{E) = then D is actually 
flat. It follows that in this case the pair [E, D) defines a representa- 
tion of TTiX in GL{n). If deg{E) 7^ 0, then the pair {E,D) defines a 
representation of ttiX in PGL(n), or equivalently, a representation of 
r in GL(n). By the theorem of Corlette (/[H^), every semisimple rep- 
resentation ofV (and therefore all semisimple representation ofrciX) 
arise in this way. 
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5. There is thus a bijective correspondence between the moduli space of 
polystable Higgs bundles of rank n and the moduli space of ( conjugacy 
classes of) semisimple representations ofV in GL(ra). If the degree of 
the Higgs bundle is zero, then the first moduli space corresponds actually 
to the representation variety for representations of HiX in GL(n). 



3.2 U(j9, -Higgs bundles 

If we fix integers p, q such that n = p + g, then we can isolate a special class 
of GL(n)-Higgs bundles by the requirements that 

E = V ®W 

where V and W are holomorphic vector bundles on X with rk(V") = p, 
rk(Vr) = g, deg(y) = a, deg{W) = b, (3 e H^{Rom{W,V) ® K), and 7 G 
H^{B.om{V, W) ^K). We can describe such Higgs bundles more intrinsically 
as follows. Let -Pgl(p) and -PGL(g) be the principal frame bundles for V and W 
respectively. Let P = Pgl{p) x PcLiq) be the fiber product, and let AdP = 
-P XAd sK^) be the adjoint bundle, where GL(p) x GL(g) C GL(n) acts by 
the Ad-action on the the Lie algebra of GL(?7,). Let {qI{p) © dKl))^ sK^) 
be the orthogonal complement with respect to the usual inner product. This 
defines a subbundle 

Pp,<i := P XAd (0l(p) © 0[(g))^ C AdP . (3.6) 
We can then make the following definition. 

Definition 3.3. A U{p, q)-Higgs bundl^ on X is a pair {P, $) where P is a 
holomorphic principal GL(p) x GL(g) bundle, and $ is a holomorphic section 
of the vector bundle Pp^g (8> K (where Pp^g is the bundle defined in (jX 



Remark 3.4- We can always write P = Pgl{p) x PcLiq)- If we let V and W 
be the standard vector bundles associated to Pgl(p) and Pchiq) respectively, 
then any $ G H^{Pp^q ® K) can be written as in (|3.5| ). We will usually adopt 
the vector bundle description of U(p, g)-Higgs bundles. 



Remark 3. 5. Definition is compatible with the definitions in and in 
There they define a G-Higgs bundle for any real form of a complex 



reductive Lie group. The bundle in their definition is a principal if -bundle. 



The reason for the name is explained by the following remarks and by Lemma 3.6 
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where if C G is a maximal compact subgroup and is its complexifica- 
tion. Thus in the case that G = U{p,q), we get that H'^ = GL{p) x GL(g). 
The Higgs field is precisely a section of the bundles which appears in Def- 
inition p.3|. From a different perspective, Definition |3.3| defines an example 



of a principal pair in the sense of ^ and ^7^. Strictly speaking, since the 
canonical bundle K plays the role of a fixed 'twisting bundle', what we get 
is a principal pair in the sense of p| . The defining data for the pair are then 
(i) the principal GL(p) x GL(g) x GL(l)-bundle -Pgl(p) x PcLiq) x Pr, where 
Pk is the frame bundle for K and (ii) the associated vector bundle Pp^g ® K. 

Lemma 3.6. Let {V © W^, $) be a \J{p,q)-Higgs bundle with a Hermitian 
metric H = Hy ® Hw, i-e. such that V Q) W is a unitary decomposition. 
Let A be a unitary connection with respect to H , and let D = dA + d be the 
corresponding Higgs connection, where 9 = $ + $*. Then D is a U(p, g)- 
connection, i.e. in any unitary local frame the connection 1-form takes its 
values in the Lie algebra of\J{p,q). 

Proof. Fix a local unitary frame (with respect to H = Hy © H^). Then 
D = d + A + 6, where A takes its values in u(p) © u(g) C u(p, q) C u{n), 
while 9 takes its values in (u(p) © u{q))^ fl u{p, q). □ 



Definition 3.7. Let {E, $) be a U(p, q)-Higgs bundle with E = V ®W and 
$ = (° o)- We say $) is a stable \]{p,q)-Higgs bundle if the slope stabil- 
ity condition l \3. i.e. fi{E') < fi{E), is satisfied for all ^-invariant subbun- 
dles of the form E' = V ® W, i.e. for all subbundles V C V and W CW 
such that 

13 -W — >V' ®K (3.7) 
7 : 1/' — >W' ®K . (3.8) 

Semistability fori] {p,q)- Higgs bundles is defined by replacing the above strict 
inequality with a weak inequality, and polystability means a direct sum of sta- 
ble \J{p,q) -Higgs bundles all with the same slope. In particular a polystable 
l]{p,q)-Higgs bundle is the direct sum of (lower rank) U{p',q')- Higgs bun- 
dles. We shall say that a polystable \]{p,q)-Higgs bundle which is not stable 
is reducible. Two \J{p, q)-Higgs bundles {V © PV, $) and {V © W',^') are 
isomorphic if there are isomorphisms gy '■ V V and gw '■ W W' which 
intertwine $ and ^' , i.e. such that {gy®gw)®lK°^ = ^'°{gv®gw) where 
Ik is the identity on K. 
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Remark 3.8. The stability condition for a U(p, g)-Higgs bundle is a priori 
weaker than the stability condition given in Definition ^TT] for GL(n)-Higgs 
bundles. Namely, the slope condition has to be satisfied only for all proper 
non-zero ^-invariant subbundles which respect the decomposition E = V (B 
W, that is, subbundles of the form E' = V ® W with V (^V and W C W. 
However, it is shown in [|T9|, Section 2.3] that the weaker condition is in fact 
equivalent to the ordinary stability of (-E, $). 

Proposition 3.9. Let $) he a \]{p,q)-Higgs bundle with E = V ® W 
and $ = o)- Then $) is \]{j),q) -poly stable if and only if it admits 
a compatible hermitian metric H = Hy © Hw such that Hitchin's equation 
^ ) is satisfied 

Proof. Though not explicitly proved there, this is a special case of the cor- 
respondence invoked in [0] for G-Higgs bundles where G is a real form of 
a reductive Lie group. By Remark it can also be seen as a special case 
of the Hitchin-Kobayashi correspondence for principal pairs (cf. and ||27[] 
and [§]). We note finally that in one direction the result follows immediately 
from Remark |3.8| : if © W, $) supports a compatible metric such that 
( |3.2| ) is satisfied, then it is polystable as a GL(?T,)-Higgs bundle, and hence it 
is U(p, g)-polystable. □ 

Remark 3.10. This correspondence allows us, via the next theorem, to use 
U(p, g)-Higgs bundles to study representations of the surface groups ttiX and 
r into U(p, q) and PU(p, q) 

Definition 3.11. Fix integers a and h. Let A^(a, h) denote the moduli space 
of isomorphism classes of polystable \]{p,q)-Higgs bundles with deg(\^) = a 
and deg W = b. 

Proposition 3.12. The moduli space Ai{a, b) can be identified with the mod- 
uli space of \J{p,q)-Higgs bundles which admit solutions to Hitchin's equa- 
tions. It is a quasi-projective variety which is smooth away from the points 
representing reducible U(p, q)-Higgs bundles. There is an homeomorphism be- 
tween TZr{(i, b) and Ai{a, b). This restricts to give a homeomorphism between 
the subspace 7l^{a,b) of irreducible elements in 7lr{a,b) and the subspace 
Ai^{a,b) of stable Higgs bundles in Ai{a,b). 



Proof. The first statement is a direct consequence of Proposition pj.9|. The 



construction of Ai{a, b) is essentially the same as in section §9 of |33|. There 



the moduli space of G-Higgs bundles is constructed for any reductive group 
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G. We take G = GL{p) x GL(g). The difference between a U(p, g)-Higgs 
bundle and a GL{p) x GL(g)-Higgs bundle is entirely in the nature of the 
Higgs fields. Taking the standard embedding of GL{p) x GL(g) in GL(p + g) 
we see that in a GL(p) x GL(g)-Higgs bundle the Higgs field $ takes its values 
in the subspace {Ql{p)®dKl)) sKp + Q')' while in a U(p, g)-Higgs bundle the 
Higgs field $ takes its values in the complementary subspace {qI{p) Q) gl{q))^ ■ 
Since both subspaces are invariant under the adjoint action of GL{p) x GL(g), 
the same method of construction works for the moduli spaces of both types 
of Higgs bundle. 

Suppose that {E = V Q)W,^) represents a point in A4{a, b), i.e. suppose 
that it is a U(p, g)-polystable Higgs bundle, and suppose that with metric 
H = Hv © Hw Hitchin's equation ( |3.2| ) is satisfied. Rewriting the equa- 
tions in terms of the Higgs connection D = + 9, where A is the metric 
connection determined by H and 6' = $ + $*, we see that D is projectively 
flat. By Lemma ^]6| it is a projectively fiat U(p, g)-connection, and thus 
defines a point in 7lr{a, h). Conversely by Corlette's theorem |]T0[, every rep- 



resentation in Hom'''(7riX, PU(p, g)), or equivalently every representation in 
Hom^(r, U(p, g)), arises in this way. □ 

Remark 3.13. If GCD(p + q.,a + h) = 1 then for purely numerical reasons 
there are no strictly semistable U(p, g)-Higgs bundles in Ai{a,b). In this 
case M^'ia, b) = M{a, b). 

Proposition 3.14. With n = p + q and d = a + b, let A4{d) denote the 
moduli space of polystable GL{n)-Higgs bundles of degree d. Ifp ^ q or a ^ b 
then J^{a,b) embeds as a closed subvariety in M.[d). If p = q and a = b, 
then there is a finite morphism from M.{a, a) to A4{d). 

Proof. Let [V © 1^, $]p,q denote the point in A^(a, 6) represented by the 
U(p, g)-Higgs bundle [v ®W,^). Then {E = V ®W,^) is a polystable 
GL(n)-Higgs bundle and the map A4{a, b) M.{d) is defined by 

[V^©iy,$]p,,^ , 

where [, denotes the isomorphism class in M.{d). The only question is 
whether this map is injective. Suppose that {E = V Q) W,^) and (£" = 
V'(BW', $') are isomorphic as GL(n)-Higgs bundles. Let the isomorphism be 
given by complex gauge transformation g : E ^ E' . Since we can regard the 
smooth splitting of E as fixed, we see that unless V = W and W = V, the 
gauge transformation must already be of the form ( ^o' ), i.e. [\^©VF, ^]p,q = 
[V © W, But in order to have V = W and W = V we require 
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p = q and a = b. In that case, if V and W are non-isomorphic, then 
[V ® W, {'^ '^)]n = [W ® V, {pi )]n but the Higgs bundles are not isomorphic 
as U(p, g)-Higgs bundles. □ 



3.3 Deformation theory of Higgs bundles 

A main tool in the study of the topology of moduli spaces of Higgs bundles is 



given by the Morse theoretic techniques introduced by Hitchin In order 
to use these methods, we first need to recall the deformation theory of Higgs 
bundles. We refer to the paper by Biswas and Ramanan for details. Let 
(i?, $) be a stable U(p, g)-Higgs bundle in J^'^{a,b). Some times it will be 
convenient to use the following notation: 

U = End(E) 
f/+ = End(\/) ©End(W), 

U- = Rom{W, V) © Rom{V, W). (3.9) 

Clearly, U = U+ ® U'. Note that $ G H°{U- (g) K) and that ad(<l>) in- 
terchanges U'^ and U~ . As it is shown in the Zariski tangent space 
to A^^(a,6) at the point defined by $) can be identified with the first 
hypercohomology of the complex of sheaves 

C -.U^ ^^U- ®K. (3.10) 
One has the long exact sequence 

— ^W{C') — ^ H^{U+) — ^ H^{U-®K) — ^ n\C') 

— > H\U+) — ^ H\U-®K) — y U\C') — y 0, 

(3.11) 

from which one obtains the following. 

Proposition 3.15. The moduli space of stable \J{p,q)-Higgs bundles is a 
smooth complex variety of dimension 1 + {p + qY{g — 1). 

Proof. Let {E, $) be a stable U(p, g)-Higgs bundle. Then {E, $) is simple, 
that is, its only automorphisms are the non-zero scalars. Thus, if {E, $) is 
stable, 

ker(ad(<l>) : H^{U) H^{U ® K)) = C. 
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Since U = © U and ad($) interchanges these two summands it follows 
that 

ker(ad($) : H\U+) H%U- K)) = C (3.12) 
ker(ad($) : H\U-) H\U+ o i^)) = (3.13) 

Hence, if (E, $) is stable, ( CT ) shows that m^{C') = C. To show that the 
moduli space is smooth at a neighbourhood of {E, $) we need to show that 
H^(C*) = 0. But we have natural ad-invariant isomorphisms = (f/^)* 
and U- ^{U-y. Thus 

ad(<l>) : H\U~^) H\U~ ® K) 



is Serre dual to ad(<l') : if°(f/-) ^ H^{U+ K). Thus (CT) shows that 
EI^(C*) = 0. The dimension of the moduli space is hence 

dimH^(C") = I - xiU^) + xiU~ ® K) 

= l + {p' + q^){g-l) + 2pq{g-l) 

= l + {p + qf{g-l). 

□ 

Remark 3.16. Notice that the dimension of the moduli space of stable U(p, q)- 
Higgs bundles is half the dimension of the moduli space of stable GL(p+g, C)- 
Higgs bundles. 

Remark 3.17. As pointed out previously, if GCD{p + q, a + b) = 1, then there 
are no strictly semistable elements in J\4{a, b) and hence Ai{a, b) is smooth. 



3.4 Bounds on the topological invariants 

In this section we shall show how the Higgs bundle point of view provides an 
easy proof of a result of Domic and Toledo |]11| which allows us to bound the 
topological invariants deg(l^) and deg{W) for which U(p, g)-Higgs bundles 
may exist. The lemma is a slight variation on the results of [1^, Section 3] 
(cf. also Lemma 3.6 of Markman and Xia p6| ). 

Lemma 3.18. Let (E,^) be a semistable U{p,q)-Higgs bundle. Then 

pilJ.{V)-f,{E))^Tki^)ig-l), (3.14) 
qifiiW)-fi{E))^im{g-l). (3.15) 

// equality occurs in ( |3.14| ) then either {E, $) is strictly semistable or p = q 
and 7 is an isomorphism. If equality occurs in ( p.l5|) then either {E, $) is 
strictly semistable or p = q and (5 is an isomorphism. 
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Proof. If 7 = then V is ^-invariant and so, by stability, fJ^iV) ^ f^{E) where 
equahty can only occur if {E, $) is strictly semistable. This proves ( p.l4| ) in 
the case 7 = and we may, therefore, assume that 7 7^ 0. Let N (1 V he 
the vector bundle associated to ker(7) and let I W be the vector bundle 
associated to im(7) K~^. Then 

rk(A^) + rk(J) = p (3.16) 

and, since 7 induces a non-zero section of det((l^/A^)* C?) / (S> K), 

deg(iV) + deg(/) + rk(/)(2^ - 2) ^ deg{V). (3.17) 

The bundles and V (B I are ^-invariant subbundles of E and hence we ob- 
tain by semistability that fi{N) ^ fi{E) and ^{V(Bl) ^ Ai(i?) or, equivalently, 
that 

deg{N) ^ l^{E)Tk{N), (3.18) 
deg(/) ^ /i(E)(p + rk(/)) - deg{V). (3.19) 

Adding ( p.l8| ) and p.l9| ) and using ( p.l6| ) we obtain 

deg{N) + deg(/) ^ 2fi{E)p - deg{V). (3.20) 

Finally, combining ( |3.17D and ( ^.2(J| ) we get 

deg{V) - THl){2g - 2) ^ 2fi{E)p - deg{V), 

which is equivalent to ( |3.14| ) since rk(7) = rk(/). Note that equality can 
only occur if we have equality in ( |3.18| ) and (|3.19|) and thus either {E, $) 
is strictly semistable or neither of the subbundles and V (B I is proper 
and non-zero. In the latter case, clearly N = and I = W and therefore 
p = q', furthermore we must also have equality in ( p. 17 ) implying that 7 is 
an isomorphism. An analogous argument applied to /? proves ( p.l5| ). □ 



Remark 3.19. The proof also shows that if we have equality in, say, ( 3.14 ) 

then '-j: V/N — > / ® -ft" is an isomorphism. In particular, if p < g and 

^i{V) - /i(E) = g-l then 7: 1/ ^ / ® fsT. 

We can reformulate Lemma p.l8| to obtain the following corollary. 

Corollary 3.20. Let (-E, $) he a semistable \J{p,q)-Higgs bundle. Then 

g(/x(E)-/i(H^))^rk(7)((7-l), (3.21) 

p{^i{E)-^i{V))^Tm{g-l). (3.22) 
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Proof. To see that ( |3.21| ) is equivalent to ( |3.14| ) one simply notes that n{W) 



fx{E) = ^{fi{E) - fx{V)). Similarly (jpl) is equivalent to ( ^TTSl) . □ 



An important corollary of the lemma above is the following Milnor-Wood 
tj^e inequality for U(p, g)-Higgs bundles (due to Domic and Toledo |1TT| , 
improving on a bound obtained by Dupont |T3[ in the case G = SU(p, g)). 
This result gives bounds on the possible values of the topological invariants 
degiy) and deg(W). 

Corollary 3.21. Let $) be a semistahle \J{p,q)-Higgs bundle. Then 

\fi{V)-fi{W)\^mm{p,q}ig-l). (3.23) 



p + q 



Proof. Since /x(E) = ^{V) + ^^f,{W) we have /i(V) -^(E) = ^(/i(V)- 
fi{W) and therefore §A§ gives 



(/i(^)-MW^))^rk(7)(^7-l). 



p + q 

A similar argument using ( |3.15| ) shows that 



i^^{W)-^^iV))^T^f3)ig-l). 



p + q 



But, obviously, rk(/3) and rk(7) are both less than or equal to min{p, q} and 
the result follows. □ 

Definition 3.22. Let a = deg(l^) and b = deg(VI^). The number 

r = ria,b) = 2'-^ (3.24) 
p + q 

is known as the Toledo invariant of the representation corresponding to 
(i?,$). 

Remark 3.23. Since 

p + q 



the inequality (|3.23| ) can thus be written 

|r| ^ min{p, q}{2g — 2) 
We denote tm = min{p, q}{2g — 2). 
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3.5 Moduli space for p = q and |r| = tm 

Suppose p = q. Then r = T{p,p,a,b) = a — b. In this section we give an 
alternative (more exphcit) description of the moduh space M.{a, b) in the case 
in which the Toledo invariant is maximal, i.e. |r| = |a — 6| = t^j = p{2g — 2). 

Before doing this, we need to review briefly the notion of L-twisted Higgs 
pairs. Let L be a line bundle. An L-twisted Higgs pair {V, 9) consists of a 
holomorphic vector bundle V and an L-twisted homomorphism 9 : V — > 
V ^ L. The notions of stability, semistability and polystability are defined 
as for Higgs bundles. The moduli space of semistable L-twisted Higgs pairs 
has been constructed by Nitsure using GIT pO). Let Aiiin, d) be the moduli 



space of polystable L-twisted Higgs pairs of rank n and degree d. 
Proposition 3.24. Let p = q and \a — b\ = p{2g — 2). Then 
M{a,b) ^ MK'2ip,a) ^ MK^ip,b). 

Proof. Let {E = V (BW,^) G A4{a, b). Suppose for definiteness that b — a = 
p{2g — 2). From ( p.l5| ) it follows that 7 : V — > W ^ K is an isomorphism. 
We can then compose P : W — > V ® K with 7 ® Id^ : V ® K — > W ^ K'^ 
to obtain a i^^-twisted Higgs pair 9w '■ W — > W ® K'^. Similarly, twisting 
/3 : W — > V ® K with K and composing with 7, we obtain a K^-twisted 
Higgs pair 9v '■ V — > V ® . Conversely, given an isomorphism 7 : V — > 
W ® K, we can recover (3 from 9v as well as from 9w It is clear that the 
(poly) stability of {E, $) is equivalent to the (poly) stability of (V, 9y) and to 
the (poly)stability of {W^9w), proving the claim. □ 

Remark 3.25. The moduli space A^x2(p, a) contains an open (irreducible) 
subset consisting of a rank N vector bundle over M^(p, a). This is because 
the stability of V implies the stability of any K^-twisted Higgs pair (V", 9y), 
and if^(End V ® K'^) = 0. The rank is determined by the Riemann-Roch 
Theorem. 



3.6 Rigidity for extreme values of the Toledo invariant 



From the bounds in Section |3.4| it follows that if p < g (a similar result holds 
for p > q) and (a, 6) such that |r| = tm there are no stable U(p, g)-Higgs 
bundles and every element in M.{a,b) is in fact reducible. In particular the 
moduli space has smaller dimension than expected exhibiting a certain kind 
of rigidity. This phenomenon (for large Toledo invariant) has been studied 
from the point of view of representations of the fundamental group by D. 
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Toledo ||36| when p = 1 and L. Hernandez when p = 2. We deal here with 
the general case which, as far as we know, has not appeared previously in the 
literature. To state our result, we use the more precise notation Ai{p, q, a, b) 
for the moduli space of U(p, g)-Higgs bundles such that deg(\^) = a, and 
deg(W^) = b, and write the Toledo invariant as 

r = T{p,q,a,b) = 2^^^. (3.25) 
p + q 

Proposition 3.26. Let {p,q,a,b) with p < q and \T{p,q,a,b)\ = p{2g — 
2). Then every element in Ai{p,q,a,b) is strictly semistahle and decompose 
as the direct sum of a polystable \J{p,p)-Higgs bundle with maximal Toledo 
invariant and a polystable vector bundle of rank {q — p)- To be precise, let 
T = p{2g — 2), then 

M{p,q,a,b) = M{p,p,a,a-p{2g -2)) x M{q - p,b - a + p{2g - 2)). 

(3.26) 

In particular, the dimension at a smooth point in A4{p,q,a,b) is 2 + {p^ + 
^q^ — 2pq){g — l), and it is hence strictly smaller than the expected dimension. 
(A similar result holds if p > q, as well as if r = —p(2g — 2) ). 

Proof. Suppose that T{p,q,a,b) = p{2g — 2). Let (£',$) G J^{p,q,a,b). 
Then fi{V) - i2{E) = ^ - 1 and ^(E) - fi{W) = ^{g - 1). Since rk(/3) and 
rk(7) are at most p, it follows from ( |3.14D and p.22| ) that rk(/5) = rk(7) = p. 
Let = im(7) K'^ and let Wp = ker(/5). One has that V ® is 
a ^-invariant subbundle of V ®W, and fj,{V © W^) = fJ.{E). We see that 
(£",$) is strictly semistable (as we already knew from Lemma |3.18| ). Since 
it is polystable it must split as 

In fact, using 

— > ker(<l>) — >V®W — >{V®W)®K — > 0. 

we see that ker $ = (0 © Wp, 0) and W/Wy = Wp. It is clear that {V © 
W^, $) e M{p,p, a, a- p{2g - 2)) and W(i e M{q -p,b- a + p(2g - 2)). 
Moreover, {V © H^-y, $) has maximal Toledo invariant, that is, T{p,p,a,a — 
p(2g — 2)) = 2p{g — 1). To complete the proof we observe that 

dim A<*(p,p, a, a - p{2g - 2)) + dimM'*(g - p,b- a + p{2g - 2)) 
= 1 + {2p)\g - 1) + 1 + {q~p)\g -l)=2 + {p' + 5q' - 2pq){g - 1), 

which, since q > 1, is smaller than 1 + {p + qY{g — 1), the dimension of 
M{p,q,a,b). □ 
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Corollary 3.27. Let {p,q,a,b) withp < q andT{p,q,a,b) = p{2g — 2). Then 
M{p, q, a, b) = Mk^ {p, a - p{2g - 2)) x M{q - p,b - a + p{2g - 2)). 



Proof. It follows from Propositions 3.26 and 3.24. □ 



4 Morse theory 

4.1 The Morse function 

An extremely efficient tool for studying topological properties of Higgs bundle 
moduli is considering the C*-action on the moduli space given by multiplying 
the Higgs field $ by a non-zero scalar. In order to explain this, it is convenient 
to consider the moduli space from the gauge theory point of view. The U(l)- 
action $ e*^$ on M.{a,b) preserves solutions to Hitchin's equations ( p.2|) 
(but the full C*-action does not preserve solutions to the first equation). 
Restricted to the smooth locus of A^(a, b) this turns out to be a Hamiltonian 
circle action with moment map (up to multiplication by a constant) 



Jx 



2 



the L -norm squared of the Higgs field. Thus, if the moduli space is smooth, / 



is a perfect Bott-Morse function (by a theorem of Frankel |jTj] ; the essential 
point is that the Morse indices are even) and the critical points of / are 
exactly the fixed points of the circle action. In particular, if (a, b) is 
smooth, then its number of connected components is the same as the number 
of connected components of the subspace of local minima of /. However, 
even if Ai{a,b) is not smooth, / can be used to obtain information about 
the connected components of Ai{a, b) in the following way. It was shown by 
Hitchin H, using Uhlenbeck's weak compactness theorem, that / is proper. 



We have the following general result. 

Proposition 4.1. Let Z be a Hausforff space and let f : Z ^ W be proper 
and bounded below. Then f attains a minimum on each connected component 
of Z and, furthermore, if the subspace of local minima of f is connected then 
so is Z . □ 

In particular this applies to our situation. 

Proposition 4.2. Let f: A^(a, 6) — >■ M 6e defined as above. Then f has a 
minimum on each connected component ofAi{a, b). Moreover, if the subspace 
of local minima of f is connected then so is A^(a, b). □ 
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4.2 Critical points of the Morse function 



Next we recall Hitchin's method for determining the local minima of 



/. A point {E, $) is a fixed point of the circle action if and only if it is a 
system of Hodge bundles, that is, 

E = F^(B---(BE^ (4.1) 

for holomorphic vector bundles Ei such that the restriction 

:= $1^, G /f°(Hom(F„ ® K). 

To see why this is true, note that, if {A, $) represents a fixed point then there 
must be a 1-parameter family of gauge transformations g{6) taking (A,^) 
to {A, 6*^$) and this gives an infinitesimal U{p) x U(g)-gauge transformation 
ip = g which is covariantly constant (i.e. rf^V' = 0) and such that [ip, $] = 2$. 
It follows that we can decompose E in holomorphic subbundles Ex on which 
acts as iX and furthermore that $ maps Ex to -Fx+i If {E, $) is stable 
each of these components of $ is non-zero since, otherwise, [E, $) would 
be reducible. We can therefore write E = Ei (B ■ ■ ■ (B E^ where the weight 
of ip on -Ffc+i is one plus the weight of ip on E^. In general (for reducible 
Higgs bundles) {E, $) is the direct sum of such chains. Furthermore, note 
that preserves V and W and, therefore, the Ek are direct sums of bundles 
contained in V and W. The decomposition E = Fi © • • • © Em gives a 
corresponding decomposition of the bundle U = End{E) into eigenbundles 
for the adjoint action of ip'- 



m—l 



-m+1 



where = Hom(Fj, E^). We now show how to find the local minima 

of / which are represented by stable Higgs bundles. In fact, one can do much 
better: it is possible to calculate the Morse index at the critical point. It 
follows from Hitchin's calculations in |^4|, §8] (see also |jl8|. Section 2.3.2]) 



that the subspace of the tangent space at (£', $) on which the Hessian of / 
has eigenvalue — is of the following complex: 



CI : ^ f/,-+i ® K, (4.2) 



where we use the notation 



u+ = Uknu+, 
f/- = [/,nt/-. 
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with and U as defined in ( p.9| ). This description of the eigenspace of 
the Hessian of / gives rise to the long exact sequence 

— eo(Q) H%u+) H\u,^,®K) ^ m\ci) 

(4.3) 

Note that if some Fk has a non-zero component in both V and W this leads 
a decomposition of (i?, $) and so, if (£", $) is stable, the are alternately 
contained in V and VT. It follows that 

t/+ = t/fc; f/" = f/fc. (4.4) 

fc even A; odd 

Thus we see that all the eigenvalues of the Hessian of / are even and that 

C'fc : ^ U2k+i ® K. (4.5) 
Hence we have the long exact sequence 

^ e°(C'fc) H\U2k) ^ H\U2k+i ®K)^ ei(C'fc) 

^ /fi(t/2,) ^ if^(f/2fc+i ®K)^ H2(C'fc) ^ 0. (4.6) 

Note that when (E, $) is a stable critical point we have M^iC'j^) = H^(C2^) = 
for A; 7^ 0, while H2(C') = and e°(C') = C. We can therefore use 
the exact sequence ( |4.6| ) and the Riemann-Roch formula to calculate the 
dimension of M^{C*,^): 

dim(Hi(C-,)) = x{U2k+i ®K)- xiU2k) 

= {g-l) (rk(f/2fc+i) + rk(f/2fe)) + deg(?72fc+i) - deg(?72fc) (4.7) 

for A; 7^ 0, while for /c = we obtain 

dim(e^(C*)) = l + {g-l) (rk(f/i) + rk(f/o)) + deg(t/i) - deg([/o). (4.8) 

In particular we can calculate the Morse index of / at a smooth critical point: 

(N m-l 
= '£{{9-l)MUk) + {-lf^Ueg{U,)). (4.9) 
fc^l ^ fc=2 

Note that this is the complex dimension; thus the Morse index is twice this 
number. From this discussion we see that the following proposition holds. 
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Proposition 4.3. A stable \]{p, q)-Higgs bundle of the form corre- 
sponds to a local minimum of f if and only if 



2k) 







for all k ^ 1. 



Definition 4.4. Let 

M{a,b) = {{E,(^) e M{a,b) \ (3 = or 7 = 0}. 



(4.10) 



The following is an important result for our approach that characterizes 
the local minima of /. 

Theorem 4.5. Let (-E, $) be a polystable l]{p,q)-Higgs bundle in A4{a,b). 
Then (-E, $) is a local minimum of f : Ai{a,b) M if and only if $) 
belongs to jV{a, b). 



Proof. This follows directly from Propositions [4.10| , [4.11| , |4.13| , and |4.15| , 
which are given in the following three sections. □ 



Remark 4 ■ 6. This Theorem was already known to hold when p,q ^ 2 (by 



the results of |jT9[, Hitchin p3[, and Xia @]), and also when p = q and 



{p - l){2g - 2) < |r| ^ p{2g - 2) by Markman-Xia |26|. 

Which section actually vanishes for a minimum is given by the following. 
Proposition 4.7. Let (-E, $) eAf{a,b). Then 

(1) 'y = ^ a/p ^ 6/g r ^ 0, 

(2) (3 = 0^ a/p ^b/q^T^O, 

In particular, /5 = 7 = z/ and only if a/p = b/q (i.e. r = 0). 

Proof. The relation between the conditions on r and the conditions on a/p — 
b/q follows directly from the definition of r (cf. p.24| ). By Lemma [3.18| and 
we have 

(3 =0 ^ a/p ^ b/q 
7 =0 ^ a/p ^ b/q. 
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Since we assume that at least one of /5 or 7 must vanish, we can use the 
contrapositives of these imphcations to infer further that 

a/p > b/q ^ J = 
a/p < b/q ^ (3 = 



It remains to check that /3 = 7 = if a/p = b/q and (E,^) is polystable. 
We know that at least one of P and 7 must vanish. Suppose that P = 0. 
Then ^{W) = C W ® K, i.e. W is ^-invariant. By stability, we get that 
fx{W) ^ /i(-E), with equality if and only if $) splits as (V,^) © {W,0). 
But fi{W) = ii{E) since a/p = b/q. Thus (£',$) splits as indicated, and 
hence V must be ^-invariant. Since ^{V) = 'y{V) G W ^ K, it follows that 
7 = 0. A similar argument shows that /3 = if 7 = 0. 

This can also conveniently be seen from the gauge theory point of view: 
Hitchin's equations show that the value of / on a U(p, g)-Higgs bundle such 
that /3 = is / = ||7||2 = q{f,{W) - while / = = pifxy - ^E)) 

if 7 = 0. □ 



Corollary 4.8. If a/p = b/q thenM{a,b) = M{p,a) x M{q,b). 

Proof. If a/p = b/q, then any (E, $) G J\f{a, b) has E = V ®W and $ = 0. 
The polystability of {E, $) is thus equivalent to the polystability of V and 
W. □ 



4.3 Local minima and the adjoint bundle 

In this section we find a criterion for [E, $) to be a local minimum in terms of 
the adjoint bundle. We use the notation introduced in Section |4.2| . Consider 
the complex C* defined in (|4^ ) and let 



X{CI) = dimH°(C,-) - dim (C,-) + dim tf (C,*) 



Proposition 4.9. Let {E, $) be a stable U(p, q)-Higgs bundle which is a fixed 
point of the -action on Ai{a, b). Then x{C*) ^ and equality holds if and 
only if 

ad(<l>) : f/+ U,^, ® K 

is an isomorphism. 
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Proof. For simplicity we shall adopt the notation 

The key fact we need is that there is a natural ad-invariant isomorphism 
U ^U* under which we have U+ ^ ^ (U-)* and ^ (U^)*. 

Since ad($)* = ad($) ® Ix-i under this isomorphism we have 

($^)* = <f!,_i®U-i. (4.11) 

We have the short exact sequence 

— . ker($+) ^ (U,^, ® KY im(<|.+) ^ 0. 

From ( 4.11| ) we have ker($^'*) = ker($I^_-^) ® K^^. Thus, tensoring the 
above sequence by K, we obtain the short exact sequence 

ker($:,_,) ^ {U,_,,r im($+) ® 0. 

It follows that 

deg(im($+)) ^ deg(f/,-+i) + {2g - 2) rk(<|.+) + deg(ker($:,_i)). 
Combining this inequality with the fact that 

deg(f/+) ^ deg(ker(<l>+)) + deg(im(<l>+)), (4.12) 

we obtain 

deg(f/+) ^ deg(f/,-^i) + {2g - 2) rk($+) + deg(ker(<|.:,_,)) + deg(ker($+)). 

(4.13) 

Since (-E, $) is semistable, so is the Higgs bundle (End(-E'), ad($)). Clearly 
ker($^) C End(i?) is ^-invariant and hence, from semistability, 

deg(ker(<l>^)) ^ 0, 

for all k. Substituting this inequality in (|4.13| ), we obtain 

deg([/+) ^ deg(f/,-^J + {2g - 2) rk($+). (4.14) 

From the long exact sequence ( [4.2|) and the Riemann-Roch formula we obtain 

X{CI) = - x{U^+, ® K) 

= {l-g) [MUD + rk(f/,-_,J) + deg(t/+) - deg(f/,-^J. 

Using this identity and the inequality ( [4.14| ) we see that 

X{CI) ^{g- l)(2rk(<|.+) - rk(t/+) - rk(t/-^J). 

Hence x{C') ^ 0. Furthermore, if equality holds we have 

rk($+) = rk(t/+) = rk(f/,-^J 

and also equality must hold in ( |4.14|) and so deg(im($^)) = deg{Ui7_^_^ K), 
showing that $^ is an isomorphism as claimed. □ 
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4.4 Stable Higgs bundles 



In this section we shall prove Theorem [4.5| for stable Higgs bundles. The 
reducible (polystable) ones will be dealt with in the next section. We continue 
to use the notation of Section K2i 



Proposition 4.10. Let $) be a stable U{p,q)-Higgs bundle with j3 = 
or 7 = 0. Then {E, $) is a local minimum of f . 

Proof. If /5 = 7 = then clearly {E, $) is an absolute minimum of /. Other- 
wise such a Higgs bundle is a Hodge bundle of length 2, that is, E = Fi®F2 
with Fi = V and F2 = W [li (3 = 0) or vice- versa (if 7 = 0). Hence 
End(E) = f/_i ®Uq® Ui, in other words, Uk = for \k\ > 1. It follows that 
the complex C* is zero for any k > and hence all eigenvalues of the Hessian 
of / are positive. □ 

The hard part is to show that any other critical point is not a local minimum 
of/. 

Proposition 4.11. Let (E, $) = (Fi © ■ ■ ■ ©F^, $) be a stable \J{p,q) -Higgs 
bundle representing a critical point of f such that m ^ 3. Then {E, $) is not 
a local minimum of f . 

Proof. Note that Uk = ^ for \k\ ^ m; in particular Um = 0. Note also that, 
since (£',$) is stable, W{C'^_i) = If'{C:^_^) = (cf. the discussion at the 
end of Section 1373) and therefore 



= -xic'^-i)- (4.15) 

We shall consider the cases when m is odd and even separately. The case 
m odd. In this case m — 1 is even and therefore ([4.4|) shows that U^_i = 
Um-i 7^ while U~ C Um = 0. It therefore follows from Proposition ^79|that 
x{C^_i) < 0. Hence we have 

Il\C'^^^) = -xiC'm^i) > 0, 

showing that {E, $) is not a local minimum of /. 

The case m even. In this case we shall show that M^{Cm-2) 7^ 0- First 
note that ( [4.4|) shows that = Um-2 and U~_i = Um-i- Thus, using 

( |4.15| ) and Proposition |^^, we get that EI^(C^_2) vanishes if and only if 

ad(<l>) : Um-2 f^m-i ® K 
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is an isomorphism. Note that 

Um-i = Hom(Fi,F„) 

f/„_2 = Hom(Fi, © Hom(F2, FJ. 

If ad($) : Um-2 Um-i -ft' is an isomorphism, the same is true for its 
restriction to any fibre. But, by stabihty of (£',$), the bundles Fi, F2, 
Fm-i and Fm are all non-zero and, therefore. Lemma [4.12| below shows that 
ad($) : Um-2 — ^ Um-i ® K cannot be an isomorphism. □ 

Lemma 4.12. Let c' : —>■ Y[ and c" : V2 ~^ ^1 linear maps between 
finite dimensional vector spaces. Assume that Y'l © V2 7^ and V'/ © V2 7^ 0. 
Define 

C : Hom(Vi , Y[) © Hom(V2, ¥3) — > Hom(V2, Y[) 

// C is an isomorphism, then exactly one of the following alternatives must 
occur: 

(1) v; = V'2' = and c' = c" = 0. 

(2) V'/ = 0, V'l, V'2, V^' ^ and c' : V'2 ^ V;. 

(3) V^ = 0, V'l, V'/, V^' 7^ and c" : V^' ^ V'/. 

In particular, ifY[, ¥3, V'/ and V'2' are all non-zero then C cannot be an 
isomorphism. 

Proof If (c', c") = (0, 0) then C = and therefore 

Hom(V'2', V'l) = Hom(V;', V'l) = Hom(V2, V'2) = 0. 

If Y'l 7^ then V'/ = V'2' = which is absurd, hence = 0. Similarly one 
sees that V'2' 7^ and thus alternative (1) occurs. Henceforth assume that 
(c',c") ^ (0,0). Let r'i = dimV^ and rf = dimV^' for i = 1,2. If C is an 
isomorphism then r'(r[ + r2'r2 = ri^r'^ from which it follows that 

r'^{r[ - r',) = r'lr[, 
1 1 II ii \ II I 

^1(^2 -^1) = ^^2^2- 
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Hence 



r'l^r^, (4.16) 
r^' ^ r'l. (4.17) 



Assume that we have strict inequahty in ( [4.16|) and ( |4.17|) . Then, in par- 



ticular, im(c') and ker(c") must both be non-zero. Choose a complement to 
im(c') in Y'l so that 

V'l = im(c') © im(c )^. 

We then have an inclusion 

Hom(ker(c"),im(c')^) ^ Hom(V2,V;). 
Let ip = {ipi,ip2) e Hom(V'/, V'l) © Hom(V^', V^) and x G ker(c"), then 

C(V')(x) = C1p2{x) -'4>ic{x) = Cll>2{x)-, 

which belongs to im(c'). Hence im(C) and Hom(ker(c"), im(c')"'") have trivial 
intersection and, therefore, C cannot be an isomorphism, which is absurd. It 
follows that equality must hold in at least one of the inequalities (|4.16| ) and 
( |4.17| ). Suppose that equality holds in ( [4. 161 ) then r'lr[ = 0. Suppose first 



that r[ = 0, i.e. V'^ = 0, then ^ and, since c' = 0, we also have c" ^ 0. 
It follows that V'2 7^ 0, V'2' and that C(V'i,^2) = -V'lc". Since C is an 
isomorphism this shows that alternative (2) occurs. In a similar manner one 
sees that if equality holds in ( [4.17|) then alternative (3) occurs. Obviously 
the three alternatives are mutually exclusive. □ 



4.5 Reducible Higgs bundles 

In this section we shall finally conclude the proof of Theorem |4.5| by showing 
that it also holds for reducible Higgs bundles. First we shall show that a 



reducible Higgs bundle which is not of the form given in Theorem 4.5 cannot 



be a local minimum of /; for this we use an argument similar to the one 



given by Hitchin [^, §8] for the case of PSL(?t,, 



Proposition 4.13. Let (-E, $) he a reducible l]{p,q)-Higgs bundle. If j3 
and 7 7^ then {E, $) is not a local minimum of f . 

Proof. We know that {E, $) is a direct sum of stable U(p', g')-Higgs bundles 
of lower rank. Since f{E, $) is the sum of the values of / on each of the 
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summands (on the corresponding lower rank moduli space), it follows that 
each of these stable summands is a local minimum of / on the lower rank 
moduli space. In particular, [E, $) is a fixed point of the circle action. 
Therefore, by Proposition [4.10| , each stable direct summand has /9 = or 
7 = 0. Hence we can choose two stable direct summands {E', $') and (£"', $") 
such that 7' 7^ and P" 7^ and = 7" = 0. It is clearly sufficient to show 
that {E' © E", $' © $") is not a local minimum of / on the corresponding 
moduli space and we can therefore assume that {E, $) = {E' © E", $' © $") 
without loss of generality. 

Let C denote the configuration space of all solutions to Hitchin's equations 
and let {Aq, $0) € C be the gauge theory object corresponding to {E, As 
in Section ^]2| we denote hy ip = g the infinitesimal gauge transformation 
counteracting the circle action. We can write = tp' + ip" , where ip' and ip" 
are infinitesimal gauge transformations of {E', and {E", $") respectively; 
recall that we may assume that tT[ip') = trip" = 0. The calculations of ||2^ , 
§8] show that if a(t) = (At, $t) is a smooth curve in C such that {Aq,^o) 
represents a fixed point of the circle action then (/oa;)'(0) = 0. Furthermore, 
if a'(0) = {A, $) is a tangent vector to C at {Aq, $0) and ip acts on A with 
weight A and on $ with weight u then 

(/oa)"(0) = -Aci-(z/-l)c2 

for strictly positive constants Ci and C2. 

Suppose that we have a family {Et, $t) of polystable Higgs bundles such 
that (i?o, $0) = (E,^) and let a{t) = {At,^t) be the corresponding family 
of solutions to Hitchin's equations. Then the holomorphic structure on Et 
is given by the (0, l)-part of A, while the holomorphic $ coincides with its 
gauge theory counterpart and hence the weight of tp on [A, $) coincides with 
its weight on {E, $). 

This provides a criterion for proving that a given fixed point is not a min- 
imum: it will be sufficient to find a 1-parameter family {Et, of polystable 
Higgs bundles such that {Eq,^q) = (-E, $) and such that (-E, $) lies in a 
direct sum of strictly positive weight spaces for ip. In fact we shall construct 
a family such that $ = and thus we simply need to consider the tangent 
vector E G H^{End{E)) at to the family Et. Recall from Section [4.2| that 



we have decompositions E' = ^ Fl. and E" = @ F^ into eigenspaces of ip. 
Clearly we have 

F[ = V' , F^ = W' , 

F[' = W" , F^' = V" . 
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Let Ay and A'^ be the weights of the action of if) on V and W respectively, 
and analogously for E" . We then have that 

^'w = -^y + 1 ! ~ + -'- ■ 

and, since ti^ip)' = tT{4')" = 0, 

Kp" + = , 

where p' = ikiV'), q' = ikiW'), p" = rkiV") and q" = ikiW"). From these 
equations we conclude that 

AW - X'w = -7^, + -/-T, > ' (4-18) 
p + q' p" + q" 

K-K = ^rrT: + ^,>^- (4-19) 

p" + q" p' + q' 

Hence the subspaces H^{}lom{W'\ W')) and H^{Y{om{V\ V")) of /fHEnd(E)) 
have strictly positive weights and it will suffice for us to construct a family 
{Et, as described above such that E lies in the direct sum of these two 
spaces and $ = 0. For this argument to be valid it is of course essential 
for this direct sum to be non-zero; in fact it is proved in Lemma [4.14| be- 
low that both if^(Hom(PF", PF')) and H''(Y{om{V' ,V")) are non-vanishing. 
In order to find such a family we adapt the construction of such a family 
used in the proof of Proposition 4.3 of |jl9[ (for SU(2, 2)-Higgs bundles): Let 
7] G H^{Kom.{y\V")) and a G H^iYiomiW" ,W)) be non-zero; we can then 
define a deformation of {E, $) by using that t] defines an extension 

— ^V" — — — > , 

while a defines an extension 

— >W' — >W'' — >W" — > . 

Let = V'^ and define by the compositions 



Note that (E^, <l>°) = {E, $) (the Higgs fields agree since b' = c" = 0). Now 
define the family {Et, = {E^vt,at) ^^(r,t,at)y ^Y^en clear that E = (r], a). 
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In order to see that $ = we simply note that our definition of <|)(^''^) did not 
change the Higgs field but only the holomorphic structure on the bundles: 
thus the reason why the Higgs field stays holomorphic with respect to the 
deformed holomorphic structure is that = 7" = 0. 

In order to conclude the proof we still need to show that [Ef, $t) is a 
polystable Higgs bundle for each t. It is in fact easy to see that {E"^'", ^^'") 
is stable: the essential point is that the destabilizing subbundles V and W" 
of {E, $) are not subbundles of the deformed Higgs bundle. We leave the 
details to the reader. □ 

Lemma 4.14. The cohomology groups (}lom{V' , V")) and (RomiW" , W')) 
are both non-vanishing. 

Proof. Since 7" = 0, V" is a ^-invariant subbundle of E" and therefore 
fi{V") < fi{E") = ^{E'). Using the Riemann-Roch formula we therefore 
obtain 

/i°(Hom(\/', V") - h\mm{V', V") = p'p"{l -g + fi{V") - fi{V')) 

<py{l-g + ^{E')-f,{V')). 



Since rk(/5') ^ p' the inequality ( |3.22|) of Corollary |3.20| shows that n{E') — 
fi{V') ^ g — 1 and we therefore deduce that 

h%Rom{V', V") - h\Rom{V', V") < 0, 

from which it follows that H^{B.om(y' ,V") 7^ 0. Similarly one sees that 
H\Rom{W",W')) ^0. □ 



In order to finish the proof of Theorem [4.5| we only need to show that any 



reducible U(p, g)-Higgs bundle with /? = 0or7 = 0isa local minimum of /. 

Proposition 4.15. Let {E, $) be a reducible U{p, q)-Higgs bundle with f3 = 
or 7 = 0. Then {E, $) is a local minimum of f . 



Proof. Recall from ( |4.10| ) that A/'(a, b) is the subspace of polystable U{p, q)- 



Higgs bundles in M.{a,b) with /5 = or 7 = 0. From Proposition |4.7| we 
know that if ajp > bjq then /3 = and f{Af{a, b)) = q{^(W) — fi{E)). Since 
we know that A4{a, b) \ Ar(a, b) does not contain any local minima of / and 
that / has a global minimum on M.{a, b) it follows that this global minimum 
is exactly f{J\f{a,b)) and, therefore, f{J\f{a,b)) consists of local minima of 
/. Of course a similar argument applies if a/p < b/q. 
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Finally, if a/p = b/q then we know from Proposition that $ = and 
hence f{M{a,h)) = 0, showing that M{a,h) consists of (global) minima of 
the positive function /. □ 



4.6 Local minima and connectedness 

Denote by A^''(a, h) C (a, h) and Af'^i^a, h) C A/'(a, h) the subspaces consist- 
ing of stable U(p, g)-Higgs bundles, and denote by M.'^{a, h) and J\f'^{a, h) their 
respective closures. In this section we explain how to obtain connectedness 
results on M.^{a, h) and M.'^{a, h). 

The invariants (a, h) will be fixed in the following and we shall occasionally 
drop them from the notation and write M. = M.{a,h), etc. 

Proposition 4.16. The closure of in M. coincides with M'^ and 

J\f' = M'nJ\f . 

Proof. Clear. □ 
Now consider the restriction of the Morse function to Ai^ , 

f: M' . 

since is closed in Ai the restriction of / remains proper. 

Proposition 4.17. The restriction of f to Ai^ is proper and the subspace 
of local minima of this function coincides with JV"^ . 

Proof. Properness of the restriction follows from properness of / and the fact 
that TW"* is closed in A4. 

We know that / is constant on Af and that its value on this subspace is 
the global minimum of / on Thus A/"** is contained in the subspace of 
local minima of /. 

It remains to see that there are no other local minima of the restriction 
of / to Ai^. We already know that the subspace of local minima on is 
A/''^. Now, A^^ is open in A<(^ so there cannot be any additional local minima 
on Ai. 

Thus all we we need to prove is that there are no local minima of / in 
(A4^ \ A^^) \A/^^ So let {E, ^) be a strictly poly-stable U(p, g)-Higgs bundle 
representing a point in this space, then from Proposition |4.16| we see that 
/3 7^ and 7 7^ 0. In the proof of Proposition |4.13| we constructed a family 
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{Et, $t) of U(p, g)-Higgs bundles such that {E, $) = {Eq, $o) and {Et, is 
stable for t ^ 0. Furthermore we showed that the restriction of / to this 
family does not have a local minimum at {Eq, $o)- It follows that {E, $) is 
not a local minimum of / on TVI*. □ 



Proposition 4.18. (1) If N'{a,h) is connected, then so is Ai{a,b). 
(2) IfN''^{a,b) is connected, then so is A^*(a,6). 



Proof. (1) In view of Proposition |4.2| , this follows from Theorem [4.5| . 

(2) If Af''^{a,b) is connected, then so is its closure A/'''(ci,&). But from 
Proposition [4.17| , Af'^{a,b) is the subspace of local minima of the proper 
positive map /: Ai'^{a,b) M. Hence the result follows from Proposi- 



tion 4.1. □ 



5 Stable triples 



5.1 Definitions and basic facts 

We briefly recall the relevant definitions for holomorphic triples as studied 
in IP and [l^; we refer to these papers for details. A holomorphic triple 
on X, T = {Ei,E2,4>) consists of two holomorphic vector bundles Ei and 
E2 on X and a holomorphic map (p: E2 ^ Ei. A homomorphism from 
T' = {E[, E'2, 0') to T = (£'1, E2, 0) is a commutative diagram 



E' 



Eo 



El. 



T' = {E[, E2,4>') is a subtriple of T = {Ei,E2,4>) if the homomorphisms of 
sheaves E[ —>■ Ei and E2 —>■ E2 are injective. For any a e M the a- degree 
and a-slope of T are defined to be 

deg„(T) = deg(Ei) + deg(E2) + ark(E2), 
degjT) 



rk(Ei) +rk(E2) 
/i(£i © £2) + a 



rk(E2 



rk(Ei) + rk(E2 
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The triple T = {Ei, E2, 0) is a-stable if 

(T') < /..(T) 

for any proper sub-triple T' = {E[, E2, 0'). Sometimes it is convenient to use 

A,(T')=/ia(T')-/i«(T), (5.1) 

in terms of which the a-stability of T is equivalent to Aq,(T') < for any 
proper sub-triple T'. We define a-semistability by replacing the above strict 
inequality with a weak inequality. A triple is called a-polystable if it is the 
direct sum of a-stable triples of the same a-slope. Write n = (ni,n2) and 
d = (di, 6/2). We denote by 

Ma =Main,d) = J\fa{ni,n2,di,d2) 

the moduli space of a-polystable triples T = {Ei,E2,(p) with rk(£'j) = rii 
and deg(£'j) = di for i = 1,2. The subspace of a-stable triples is denoted 
by J\f^. We refer to (n, d) = (ni, ^2, (ii, ^2) as the type of the triple. Like 
in the Higgs bundle case, the stability condition for triples arises in relation 
to some gauge-theoretic equations, known as the vortex equations. Namely, 
given a triple T = [Ei, E2,(f)), one is looking for hermitian metrics Hi and 
H2 on El and E2, respectively, such that 

V^AF{Ei) + <f)<p* = TildE,, 
v^AF(E2) - 0*0 = r2 Idij,, 

where A is contraction by the Kahler form of a metric on X normalized so 
that vol(X) = 27r, F{Ei) is the curvature of the unique connection on Ei 
compatible with hi and the holomorphic structure of E^, and ti and T2 are 



real parameters satisfying di + d2 = tiiTi + n2T2. A solution to (|5.2|) exists 
if and only if T is a-polystable for a = ti — T2 (0). There are certain 
necessary conditions in order for a-semistable triples to exist. Let fj,i = di/rii 
for 2 = 1,2. We define 

Olm =/^i - /i2, (5.3) 

, ni + n2 . , 
aAf =(1 + n -)[iii-ii2), ni^n2. (5.4) 



One has the following (p, |I5|). 

Proposition 5.1. The moduli space Afa{ni,n2,di,d2) is a complex analytic 
variety, which is projective when a is rational. A necessary condition for 
J\fa{ni,n2,di,d2) to be non-empty is 
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^ dm ^ a ^ if ni 7^ n2, 

^ (y-m ^ Q; if rii = 

The moduli space of triples for a = am is given by the following. 

Proposition 5.2. A triple T = {Ei,E2,4>) is am-polystable if and only if 
= and El and E2 are polystable. We thus have 

A/'a„(ni,n2,rfi,c?2) = M{ni,di) x M{n2,d2). 

Proof. Consider equations ( |5.2| ) on T. If a = am then ri = /ii and T2 = ^2 
and hence in order to have solutions of ( ^.21) we must have = 0. In this 
case, ( [5. 21 ) say that the hermitian metrics on Ei and E2 have constant central 
curvature. But this is equivalent to the polystability of Ei and E2 by the 
theorem of Narasimhan and Seshadri . □ 

Remark 5.3. If am = and ni 7^ n2 then am = oim = and the moduli 
space of a stable triples is empty unless a = 0. 

Given a triple T = {Ei, E2, (p) one can define the dual triple T* = (£"2 , El,(j)*), 
where E* is the dual of Ei and (p* is the transpose of (p. It is not diffi- 
cult to prove (0) that the a- (semi) stability of T is equivalent to the a- 
(semi) stability of T*. The map T ^ T* defines then an isomorphism 

A/'«(ni, ris, c?i, ^2) = Ma{n2, rii, -rfs, -di). 

This can be used to restrict our study to ni ^ n2 and appeal to duality to 
deal with the other case. A triple T = {Ei,E2,(f)) is strictly a-semistable if 
and only if it has a subtriple T' = {E[, E2, (j)') such that 

lJi{E[ © E'^) + a , = © E2) + a . (5.5) 

+ ^2 ^1 + n2 

There are two ways in which this can happen: the first one is if there exists 
a subtriple T' such that 

n'2 _ n2 
n[ + n'2 ni + n2 
li{E[ © E'2) = ii{Ei © E2) . 

In this case the terms containing a drop from ( |5.5| ) and T is strictly a- 
semistable for all values of a. We refer to this phenomenon as a-independent 
semistahility. This cannot happen if GCD(n2,ni + n2,di + ^2) = 1- The 
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other way in which strict a-semistabihty can happen is if equahty holds in 
(p|) but 

(5.6) 

+ ^2 rii + n2 

The values of a G [a™, oo) for which this happens are called critical values. 
For such an a there exists {n'^, n'2, d[, d'2) such that 

d\ + d'o n'2 di + d2 n2 
+ a— = h a- 



n[ + n'2 + n'2 ni + n2 ni + n2 

In other words, 

(ni + ri2)« + d'2) - (n'l + n'2)[dx + ^2) 



n'^?22 — 721^2 



with n!^ ^ nj, (n'^, ng, c?'^, c/g) 7^ (ni, n2, (ii, ^2), (^'i,''^2) 7^ (0,0) and n']^?22 7^ 
n\n!2- We say that a is generic if it is not critical. We thus have the following 
(cf. i). 

Proposition 5.4. (1) There is only a discrete number of critical values of 
a G [am, 00) for given {ni,n2,di,d2). 

(2) If ni 7^ n2 the number of critical values is finite and lies in the interval 
[am, aM]- 

(3) The stability criteria for two values of a lying between two consecutive 
critical values are equivalent; thus the corresponding moduli spaces are 
isomorphic. 

(4) If a is generic and GCD(?t,2, ni + n2,di + ^2) = 1 then a-semistability 
is equivalent to a-stability. 

Let a,^ = am + e, with e such that the interval {am, does not con- 
tain any critical value (sometimes we refer to this value of a as small. The 
following is important in the construction of the moduli space for small a. 

Proposition 5.5 (|]^). // a triple T = {Ei,E2,4>) is a^-semistable triple, 
El and E2 are semistable. In the converse direction, if Ei or E2 is stable 
and the other is semistable, T = {Ei,E2,(f)) is a^-stable. 



Corollary 5.6. // GCD(r;,i, rfi) = 1 and GCD{n2,d2) = 1, the moduli space 
A/'\ {ni,n2, di, 6/2) is isomorphic to the projectivization of a Picard sheaf over 
M{ni,di) X M{n2,d2). 
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Proof. Let Ei and E2 the universal bundles over X x M{ni,di) and X x 
M(n2, 6^2) , respectively. Consider the canonical projections vr : XxM{ni, di) x 
M(n2,(i2) M{ni,di) x M(n2,rf2); vr : X x M{ni,di) x M{n2,d2) X; 
TTi : X X M{ni, di) x M{n2, ^2) — * X x M(ni, di); and : X x M(ni, di) x 



M(n2, ^2) ^ X X M(n2, ^2)- From Proposition ^]5| we deduce that 
AP+ (rai,n2,c/i,rf2) = P(i?V,(7r*Ei ^tt^E; (g)7r*fs:)*). 

□ 

It is important for us to have criteria to rule out strict a-semistability when 
a is an integer. 

Lemma 5.7. Letm be an integer such that GCD{ni + n2,di + d2 — mni) = 1. 
Then 

(1) a = m is not a critical value, 

(2) there are no a-independent semistable triples. 

Proof. To prove (1), suppose that a = m is a critical value. There exist then 
a triple T and a proper subtriple T' so that 

{d[ + d'2 + mn2){ni + ^2) = {di + d2 + mn2){n[ + n^). 

This means in particular that ni + n2 divides {di + 6/2 + mn2){n[ + n'g) and 
since ni+n2 > n[ + ra'g we must have that GCD(ni + n2, rfi + ^2 + mr7,2) > 1. 
But, since rfi + ^2 + "^^^2 = di + d2 — mrii + m(ni + 712), we have GCD(ni + 
^2, (ii + (^2 — mrii) > 1, in contradiction with the hypothesis. To prove (2), we 
will show that GCD(n2, ni + ^2, di + (^2) = 1, from which the result follows 
by (4) in Proposition pl^ . Suppose that GCD(n2, rii +n2, di + ^2) 7^ 1- Then 
there is in'n'.d') such that = and = It follows 

that 

^1 + c^2 ~ '^'^'i (ii + (i2 — mrii 
n']^ + 712 ni + 712 

and hence GCD(ni + n2,di + d2 — mni) 7^ 1, in contradiction with the 
hypothesis. □ 
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5.2 Minima as triples 

Let {E, $) be a U(p, g)-Higgs bundle with 7 = 0. We can then define a 
holomorphic triple T = [Ei, E2, 4>) by 

Ei^V^K 
E2^W, 

and given a holomorphic triple we can define an associated U(p, g)-Higgs 
bundle with 7 = 0. Similarly, there is a bijective correspondence between 
U(p, gf)-Higgs bundles with (3 — and holomorphic triples given by 

Ei^W^K, 
E2^V 
= 7. 

The link between the stability conditions for holomorphic triples and U(p, q)- 
Higgs bundles is given by the following result. 

Proposition 5.8. A \J{p, q)-Higgs bundle (E,^) with P = or 'j = is 

(semi)stahle if and only if the corresponding holomorphic triple T = {Ei, E2, (p) 
is a- (semi) stable for a = 2g — 2. 

Proof. For definiteness we shall assume that 7 = (of course, the same 
argument applies ii P — 0). We then have Ei — V iSi K and E2 — W and, 
hence, 

deg(£;i)=deg(y)+p(2^-2). 
Since p = rk{Ei) and q = Tk{E2) it follows that 

deg(y) + deg{W) +p{2g - 2) + aq 



t^a{T) 



p + q 

^,(E) + ^{2g-2) + ^a. 
p + q p + q 



If we set q; = 2g( — 2 we therefore have 

pta{T)^pL{E) + 2g-2. (5.7) 

Clearly the correspondence between holomorphic triples and U{p, q')-Higgs 
bundles gives a correspondence between sub-triples T' — {E[, E'2, (/)') and 
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^-invariant subbundles of E which respect the decomposition E = V (B W 
(i.e., subbundles E' = V ® W with V CV and W CW). It was shown in 
Section 2.3 of |]T^ that {E, $) is (semi)stable if and only if the (semi) stability 
condition holds for ^-invariant subbundles which respect the decomposition 
E = V (B W. On the other hand, it follows from (15.71) that 



if and only if 

(T') < /i„(r) 

(and similarly for semistability), thus concluding the proof. □ 

Recall that A^(a, b) is the space of polystable \J{p, g)-Higgs bundles with 
deg{V) = a and deg{W) = b. We have the following important characteriza- 
tion of the subspace of local minima of / on A4{a, b). 

Theorem 5.9. Let J\f{a,b) he the subspace of local minima of f on Ai{a,b) . 

(1) If a/p ^b/q then 

Xia, b) = M2g-2{p, g, a + Pi^g - 2), 6), 

the moduli space of a-polystable triples with rii = p, n2 = q, di = 
a + p{2g — 2), d2 = b, and a = 2g — 2. 

(2) If a/p ^ b/q then 

X{a, b) ^ M2g-2{q, P, b + q{2g - 2), a), 

the moduli space of a-polystable triples with ni = q, n2 = p, di = 
b + q{2g — 2), d2 = a, and a = 2g — 2. 

(3) If a/p = b/q then 

Af{a,b) ^ M{p,a) x M{q,b), 

where M{p, a) is the moduli space of polystable bundles of rank p and 
degree a, and M{q, b) is the moduli space of polystable bundles of rank 
q and degree b. 

Proof. It is obtained by combining Theorem |4.5| , Proposition [4.7| , Proposi- 
tion |5.8| and Corollary W^. □ 
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Remark 5.10. If a/p = h/q then 2(7 — 2 is equal to the extreme value, am-, 
for triples of type (p, g, a + p{2g — 2), 6) and also for triples of type (g, p, 6 + 
g(2gf — 2), a). Thus by Proposition the descriptions of A/'(a, 6) in (1) and 
(2) of Theorem ^]9| coincide with each other and agree with that in (3). 



Theorem 5.11. (1) Let a/p ^ h/q. 

— If Af2g-2{p,(l,(^ + p{'^9 — 2), 6) is connected then A4{a,b) is con- 
nected. 

— If J^2g-2iPj q,a + p{2g — 2), h) is connected then M.^[a, h) is con- 
nected. 

(2) Let a/p ^ h/q. 

— If J\f2g-2{q,p,b + q{2g — 2), a) is connected then J^{a,b) is con- 
nected. 

— // A/'2*g_2(g,P, & + q{'2g — 2), a) is connected then Ai^{a,b) is con- 
nected. 

(3) If a/p = h/q then Ai{a,b) and A4^{a,b) are connected. 



Proof. It follows from Proposition [4.18| and Theorem |5.9| . For (3) we also use 
that the moduli spaces of semistable bundles M{p, a) and M{q, b) are con- 
nected, and that the moduli spaces of stable bundles M'^{p,a) and M'^{q,b) 
are connected (in fact these moduli spaces are irreducible.) □ 



In view of Theorem [5 .91 , it is important to understand where 2g — 2 lies 
in the range (given by Proposition |5.1| ) for the stability parameter a. 



Proposition 5.12. Let {ni,n2,di,d2) be the type of the triples arising from 
M{a,b), as in Theorem |5.^ . Let r he the Toledo invariant as in ( |3.24| ). 

(1) 2g — 2^ am, with equality if and only if t = 0. 

(2) If p ^ q then the Milnor-Wood inequality |r| ^ min{p,q}{2g — 2) is 
equivalent to the condition 2g — 2 ^ um- Moreover, 2g — 2 = au if and 
only if \t\ = mm{p,q}{2g — 2). 

(3) Ifp = q, then the Milnor-Wood inequality is equivalent to the condition 
am ^ 0. Moreover am = if and only if \t\ = p{2g — 2). 
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Proof. Let (-E, $) G J\f{a,b). We assume that a/p ^ b/q, i.e. r ^ (the 
case a/p ^ b/q is analogous). Then fii = b/q + 2g — 2, fi2 = a/p, and hence 
C(m = fi'i — fi'2 = b/q ~ a/p + 2g — 2. We thus have ^ 2g — 2. Hence, the 
Toledo invariant ( p.24|) is 

2pq ^a 6, 2pq ^ x 

-{2g - 2- am), (5.8) 



p + q p q p + q 

from which we see that am ^ 2(yf — 2 is equivalent to r ^ and, in particular, 
am = 2(7 — 2 if and only if r = 0. This proves (1). From (|5.8|) we see that 
am ^ is equivalent to 

r^^^(2^-2). (5.9) 

p + q^ ^ ^ ^ 

U p = q then (|5.9| ) is equivalent to r ^ pi'^g — 2), which is the Milnor-Wood 
inequality, proving (2). If p ^ q, then ( |5.9| ) is weaker than the Milnor-Wood 
inequality. But in this case au is an upper bound for a and in order to have 
existence we must assume 

2^-2 ^om- (5.10) 



Suppose that p > q (the case p < g is analogous), then au = '^'^m and 
hence (15. 81) can be written as 



-{2g-2 — aM)- (5.11, 



p + q 2p 



Hence, from ( ^.11| ) we see that ( ^.IQ ) is equivalent to r ^ q{2g — 2), and in 



particular we have 2g — 2 = aM if and only if r = q{2g — 2). Which conludes 
the proof of (3). □ 

Remark 5.13. The above proposition gives another explanation for the Milnor- 
Wood inequality in Corollary |3.21| . Using the fact that the non-emptiness of 



M.[a, b) is equivalent to the non-emptiness of either A/'23_2(p, q, a+p(25'— 2), b) 
or J\f2g-2{(l,P,b + q{2g — 2), a), we see that the Milnor-Wood inequality 
is equivalent to the condition that 2g — 2 lies within the range where a- 
polystable triples of the given kind exist. From this, we see that in order to 
study A^(a, b) for different values of the Toledo invariant, we have to study 
the moduli space of triples for a that may be lying anywhere in the a-range, 
including in the extreme values and aM- 

The rest of the paper is devoted to this question. The strategy we adopt 
is the following: When p ^ q, and assuming that 25f — 2 is not an extreme, we 
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give a explicit birational description of what we call the large moduli space. 
This is the moduli space for a~^j- = — e, for e > small enough so that 
there is no any critical value in [a^j,Q;jv/)- Once we have identified this, we 
need to study the changes that happen when we cross a critical value. We 
do this with all the critical values until we get to a = 2g — 2. When p = q 
there is no upper bound for a but it turns out that the moduh space remains 
unchanged after a certain value of a (Sec. WM) - We can thus apply the same 
principle as before taking this as the large moduli space. 

5.3 Extensions and deformations of triples 

Let V = {E[, E'^, (f)') and T" = {E'l, E'^, (p") be two triples and, as usual, let 

(n',d') = (n'i,n2,rf'i,4), 
(n",d") = K,<<,4'), 

where = deg(£;;), = deg(Ef), < = deg{El) and < = deg(Ef). Let 
Hom(T", T') denote the linear space of homomorphisms from T" to T', and 
let Ext^ (T",T') denotes the linear space of extensions of the form 

— >T' — >T — ^ T" — ^ 0. 

In order to analyse Ext^ (T",T') one considers the complex of sheaves 

C'{T", T') : E'r ®E[® ®E'^^ ® E[, (5.12) 

where the map c is defined by 

Proposition 5.14. There are natural isomorphisms 

Hom(T",T') = e°(C"(r",T')), 
Ext^(T",T') = m\C'{T",T')), 

and a long exact sequence associated to the complex C*(T",T'): 

— >lf{C'{T",r)) — > H\E';* ®E[®E'^ ®E'^) — > H'^iEf^E'^) 
^Hi(C"(T",T')) — ^ H\E'(* ®E[®E'^ ®E'^) H\E'f ® E[) 

— >m^lc'lT",T')) — > 0. 

(5.13) 
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Proof. A proof can be given by using cocycle representatives of the hyper- 
cohomology classes, in a manner similar to what is done in the study of 
deformations In fact the result is a special case of a much more general 



result proved in |20]. □ 



We introduce the following notation: 

h\T\T') = dimff(C"(T",T')), 

X(T", T') = h%T\ T) - h}{T\ T) + h^{T", T'). 

Proposition 5.15. For any holomorphic triples T' and T" we have 

X(T", T) = xiE'r ® E[) + xiET ® E',) - xiET ® E[) 
/ 1 \ f II I \ II I II I \ 



Proof. Immediate from the long exact sequence ( |5.13| ) and the Riemann- 



Roch formula. □ 

Corollary 5.16. For any extension — T' — T ^ T" ^ of triples, 
X{T, T) = x{T', T') + x(T", T") + x(T", T') + x(T', T"). 

□ 

Remark 5.17. Proposition |5.15| shows that x(T",T') depends only on the 
topological invariants (n', d') and (n", d") of T' and T". Whenever convenient 
we shall therefore use the notation 

x(n",d",n',dO = x(T",T'). 

The following vanishing results for H[° and play a central role in our 
problem. 

Proposition 5.18. Suppose that T' and T" are a-semistable. 

1. If liaiT') < fi^{T") then EP{C'{T",T')) ^ 

2. If HaiT') = HaiT") and T" is a-stable, then 



lf{C'{T",T')) 



C zfT' = T" 
ifV^ T". 
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Proof. By Proposition |;T3| we can identify H°(C"(T", T')) with Hom(T", T'). 
The statements (1) and (2) are thus the direct analogs for triples of the same 
results for semistable bundles. The proof is identical. Suppose that we can 
find a : T" — > T' is a non-trivial homorphism of triples h : T" —>■ T' . If 
T' = {E[, E'^, $') and T" = {E'{, E'^, <l>") then h is given by a pair of bundle 
maps Ui : E" E" for i = 1, 2 such that $' o ^2 = Mi o We can thus 
define subtriples of T" and T' respectively by = (ker('Ui), ker('U2), and 
Tj = (im('Ui), im(M2), $'), where in Tj, it is in general necessary to take the 
saturations of the image im(Mi) and im(M2). By the semistability conditions, 
we get 

/ia(T^) ^ l^aiT") ^ fia{Tj) ^ i^^T') . 
The conclusions follow directly from this. □ 

Proposition 5.19. Suppose that triples T' and T" are a-semistable with 
fia{T')=f^a{T")- Then 

1. H2(C"(T", T')) = whenever a>2g-2. 

2. If one ofV, T" is a + e-stable for some e ^ 0, then B.^{C'{T" ,T')) = 
whenever a ^ 2g ~ 2. 



Proof. From (|5.13|) it is clear that the vanishing of M {C'{T" ,T')) is equiv- 



alent to the surjectivity of the map 

H\E'r (S)E[(B E'^ ® E'^) H\E'^* ® E[) . 
By Serre duality this is equivalent to the injectivity of the map 

H\E[* E'J, ® K) ^ H\E[* ® E'( ® K) ® H\E'^* ® E'i ® K) 

iP I * ((0"(g)ld) O^/>,^/>O0'). 

(5.14) 

Proof of (1): Suppose that P is not injective. Then there is a non-trivial 
homomorphism ip : E[ ^ E2 ® K in kerP. Let I = imip and = kerip. 
Since (^''^Idi^-) o?/; = 0, / C ker 0" and hence Tj = (0, I^K*, 0) is a proper 
subtriple of T". Similarly, the fact that o (p' = implies that im0' C 
and thus = {kerip, E2, (j)') is a proper subtriple of T' . Let k = rk(A^) and 
/ = deg(A^). Then, from the exact sequence 

— ^A^ — ^ E[ — ^ I — ^0. 
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we see that rk(/) = n'l — k and deg(/) = d[ — I. Hence 



I + d'r, Tin 

+ a = — , 

k + n'r, k + Tin ' 



1,2 -r 

Adding these two expressions, and clearing denominators we see that 

d[ + 4 + {n[ -k){2- 2g) + a{n\ + n'^ - k) = {k + n'2)/ia(Tj^) + {n[ - fc)/ia(T/) 

But fiaiTj^) ^ fiaiT'), fiaiTj ) ^ ^a{T") and fia{T') = fia(T"). From this we 
obtain that 

d[ + 4 + W - ^)(2 - 2g) + a{n[ + n'^ - k) ^ d[ + 4 + an'2 , (5.15) 
and hence 

a{n[ -k) ^ {n[ - k){2g - 2) . 

Since n\ — k > Q get that a ^ 2g — 2. Hence P must be injective if the 
hypotheses of the part (1) of the proposition are satisfied. 

Proof of (2): Suppose that T" is a + e-stable for some e ^ 0. It follows that 

fla+e(Tj) < fla+e(T"), i.e 

n" 

/ia(r;) - /i,(r") < e(^^ - 1) ^ o . 

rii +712 

Thus, following exactly the same argument as in the proof of (1), we get a 
strict inequality in ( p.l5| ). We conclude that that if P is not injective then 
a < 2g — 2, i.e. if a ^ 2(7 — 2 then P must be injective. If T' is a + e-stable 
for some e ^ then we get that 

^^aiT'^) - < e(^^ - < . 

n[ +112 ^ + ^2 

The rest of the argument is the same as in the case that T" is a + e-stable. □ 

Corollary 5.20. Let T' and T" be two holomorphic triples. 
(1) dim Ext ^(T",T') = h\T\T') + h\T\T') - xiT",T'). 
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(2) IfT' and T" are a-semistable, ^ia{T') = fia{T"), and a > 2g — 2, then 

dim Ext i(T",T') = h\T"X) - xiT",T'). 

The same holds for a ^ 2g — 2 if in addition T' or T" is a + e-stable 
for some e ^ 0. 

Since the space of infinitesimal deformations of T is isomorphic to M.^{C*{T, T)), 
these considerations also apply to studying deformations of a holomorphic 
triple T. To be precise, one has the following. 

Theorem 5.21. LetT = {Ei,E2,<f)) be an a-stable triple of type {ni,n2,di,d2) ■ 

(1) The Zariski tangent space at the point defined by T is isomorphic to 

m\C*{T,T)). 

(2) IfM.'^{C*{T,T)) = 0, then the moduli space of a-stable triples is smooth 
in a neighbourhood of the point defined by T. 

(3) H^(C*(T, T)) = if and only if the homomorphism 

H\Ei ® El © e; ® E2) h\e; ® El) 

in the corresponding long exact sequence is surjective. 

(4) At a smooth point T G A/'^(?2i, n2, (ii, ^2) the dimension of the moduli 
space of a-stable triples is 

dimAf^{ni,n2,di,d2) = h\T,T) = l-x(T,T) 

= {g - '^){n\ + n\- ^1^2) - nid2 + n2di + 1. 

(5.16) 

(5) If (j) is injective or surjective then T = {Ei,E2,(p) defines a smooth 
point in the moduli space. 

(6) If a ^ 2g — 2, then T defines a smooth point in the moduli space, and 
hence Af^{ni,n2,di,d2) is smooth. 

Proof. Statements (1) and (2) are proved in (see also 0). (3) follows 
from the long exact sequence ( ^.13| ) with T = T' = T" . (4) follows from (1), 
(2) and Propositions |5.15| and |5.2U| . (5) is proved in Prop. 6.3]. (6) is a 



consequence of Proposition 5.19. □ 



52 



6 Crossing critical values 



In this section we study the differences between the stable loci ^^^(n, d) in 
the moduh spaces A/'a(n, d), for fixed values of n = (ni, 712) and d = {di, ^2) 
but different values of a. Since in this section n and d are fixed, we use the 
abbreviated notation 

= f^^in, d) and AT^ = ^^{n, d) . 

Our main result is that for all a ^ 2g — 2 any differences between the A/'q are 
confined to subvarieties of positive codimension. In particular, the number of 
components and the irreducibility properties of the spaces A/"^ are the same 
for all a satisfying a ^ 2g—2 and am < a < a^^. If the coprimality condition 
GCD(n2, rii + n2,di + = 1 is satisfied, then Af^ = Afa at all non-critical 
vales of a, so the results apply to Afa for all non-critical a ^ 2g — 2. 

We begin with a set theoretic description of the differences between two 
spaces Afa-^ and Af^^ when ai and 0:2 are separated by a critical value (as 
defined in section |5.1| ). For the rest of this section we adopt the following 
notation: Let ac be a critical value such that 

a^n < ac < au ■ (6.1) 

Set 

a'^ = Uc + t , a~ = Uc — e , (6-2) 

where e > is small enough so that is the only critical value in the interval 
(«-,«+). 



6.1 Flip Loci 

Definition 6.1. Let G [oimiOiM) he a critical value for triples of type 
(n, d). We define flip loci S^± C Af^± by the conditions that the points in 
represent triples which are a'^ -stable but a~ -unstable, while the points 
in iSq,- represent triples which are a~ -stable but -unstable. 

Remark 6. 2. The definition of S„+ can be extended to the extreme case = 
am- However, since all a^-stable triples must be a~-unstable, we see that 
S + = Af'^+ . Similarly, when ni 7^ ^2 we get S - = Af^- . The only 
interesting cases are thus those those for which am < «c < ctM- 



■^When ni 7^ 712 the bounds and ajv/ are as in (5.3) and ( ^.3[ ). When ni = 712 we 
adpot the convention that um = 00 
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Lemma 6.3. In the above notation: 

Af% - = Af^ = Af'- - S- . (6.3) 



Proof. By definition we can identify Af^+ — = Af^- — S^^ 

Suppose now tliat t is a point in Af'^+ — 5„+ = Af'^_ —S- , but tliat t is not 
in J^^c- T be a triple representing t. Tfien T lias a subtriple T' (1 T for 
wfiicli fiaciT') ^ fiacC^), and also fi^±(T') < fi^±{T). This is not possible, 
and hence t G Af^ . 

Finally, suppose that t G Af^^ and let T be a triple representing t. Then 
l^a^T') < fJ^adT) for all subtriples T' C T. But since the set of possible 
values for HadT') is a discrete subset of M, we can find a 5 > such that 
l^adT') -f^aST) ^ -5 for all subtriples T' C T. Thus /i„± (T') - /i„± (T) < 0. 
That is, t is in Af^±, and hence A/"^ C Af^^ -Sa±. ' ' □ 

Our goal is to show that the flip loci S^± are contained in subvarieties of 
positive codimension in Af^± respectively. 

Proposition 6.4. Let ac G (am, om) be a critical value for triples of type 
(n, d) = (ni, n2, di, (^2). Let T = {Ei, E2, (f)) be a triple of this type. 

1. Suppose that T represents a point in S^+, i.e. suppose that T is a^- 
stable but a~ -unstable. Then T has a description as the middle term 
in an extension 

^ T' ^ T ^ T" ^ (6.4) 

in which 

(a) T' and T" are both a'^-stable, with /i^+(T') < /i^+(T), 

(b) T' and T" are both ac-semistable with fiac{T') = /ia^(T), 

n' 

(c) is a maximum among all proper subtriples T' G T which 
satisfy (h), 

(d) n'^+ n'2 is a minimum among all subtriples which satisfy (c). 

2. Similarly, ifT represents a point in , i.e. ifT is a~ -stable but a^- 
unstable, then T has a description as the middle term in an extension 

~M in which 



(a) T' and T" are both -stable with fi^-{T') < fi^^(T), 

(b) T' and T" are both ac-semistable with /Xq,^(T') = //^^(T), 
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(c) J^^^i is a minimum among all proper subtriples T' a T which 
satisfy {h), 

(d) n[ is a minimum among all subtriples which satisfy (c). 

Proof. In both cases (i.e. (1) and (2)), since its stability property changes 
at etc, the triple T must be strictly ac-semistable, i.e. it must have a proper 
subtriple T' with /^^^(T') = Ha^{T). We can thus consider the (non-empty) 
set 

T,^{T'CT\ li^XT') = liaST) }■ 
{Proof of (1)) Suppose first that T is ct^-stable but aj-unstable. We observe 
that if T' e J^i, then , < . since otherwise T could not be Q;j"-stable. 

n\+n2 ni+n2 ' c 

But the allowed values for j^r^ are limited by the constraints ^ rii, 
^ 712 ^ ^2 and n'l + n'2 0. We can thus define 



Ao = max 

and set 



^2 



n'l + n'2 



T' e j^i 



n'2 . 

An 



n'l + n'2 

Now let T' be any triple in ^2- Since T' has maximal ac-slopc, we can 
assume that T" = T/T' is a locally free triple, i.e. if T" = (£'2 , -E" , then 
E2 and are both locally free. Furthermore, since T is Oc-semistable and 
liac{T') = iJ'adT) = iJ,ac{T"), it foUows that both T' and T" are ttc-semistable 
and of the same ctc-slope. We now show that T" is a^-stable. Suppose not. 
Then there is a proper subtriple f" C T" with li^+{f") ^ //«+(T"). However, 
since we can assume that is not a critical value for triples of type (T"'), 
we must have 

Thus, since {T") is ctc-semistable, we must have 

If fj.a^{T") < fj^adT"), say fj^adT") = Hac{T") — S, then in order to have 
/^a+(n>/^a+(n we must have 

n'n n'n S 
> h 



n'l + n'2 n'l + n'2 e 
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Letting e approach zero, we see that -„^-„ must be arbitrarily large. This 
cannot be if ^ h'l ^ n'{ and ^ 77,3 ^ n'J^ (and h'l + 77,2 > 0). We may thus 
assume that ^^^{T") — HadT"). Consider now the subtriple T' C T defined 
by the pull-back diagram 

This has iiadT') = /^.^^(T") = HadT) and thus 



— ^ Ao 



n'l + h'2 n'l + n'2 

It follows from this and the above extension that 

-// , -// ^ Ao = — ^— — Y ■ 

n{ + '^l + "^2 

However, since HadT") — jJ-adT) but ijl^+{T') < /i^+(T), we have that 

< 



Combining the previous two inequalities we get 



fin n'o 



< 



h'l + ^2 n'( + 772 

which is a contradiction. Now take T' G J-'2 with minimum rank (i.e. mini- 
mum n[ + n2) in T2- Wc claim that T' is a^-stable. If not, then as before it 
has a proper subtriple T' with 

n'2 ^ n'r, 



But then n[ + ni^ < n'l + n2, which contradicts the minimality of 77^ + 772. 
Thus T' is a^-stable. Moreover, since T is a^-stablc it follows that fi^+ [T') < 
yU^+(T). Thus taking T' G with minimum rank, and T" = T/T', we get a 
description of T as an extension in which (a)-(d) are satisfied. 

Proof of (2). If T is Q;~-stable but a^-unstable, then J^^^, > ^^"^^^ for all 
T' ^ T\. The proof of (i) must thus be modified as follows. With 



An = min 



n!n 



n\ + n'n 



T' G 
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we can define 



Ao 



and select T' G J^2 such that T' has minimal rank in J^2- It follows in a 
similar fashion to that above that T has a description as 

^ T' ^ T ^ T" ^ 

in which all the requirements of the proposition are satisfied. □ 

Remark 6.5. Unlike for Jordan-Holder filtrations for semistable objects, the 
filtrations produced by the above proposition are always of length two, i.e. 
always yield a description of the semistable object as an extension of stable 
objects. This is achieved by exploiting the extra 'degree of freedom' provided 
by the parameter ac- The true advantage of never having to consider exten- 
sions of length greater than two is that it removes the need for inductive 
procedures in the analysis of the flip loci. 

Definition 6.6. Let ac G (a^, a^) be a critical value for triples of type 
(n,d). Let (n',d') = {n[,n'2,d[,d'2) and (n",d") = {n'{,nl,d'{,d'i) he such 
that 

(n, d) = (n', d') + (n", d") , (6.5) 
(i.e. ni = n'l + n'(, n2 = n'2 + di = d[ + d'(, and 1^2 = + ^-2), and also 

d[ + d'2 n'2 d'[ + d'^ n'2' , , 

— + Oic—, T = —Ti T, + (^c—r, (6.6) 

Ui -h n2 n^ + n2 ni + ni + 

1. Define S^+{n",d",n',d') to be the set of all isomorphism classes of 
extensions 

— ^ T' — — ^ T" — ^ 0, 

where T' andT" are -stable triples with topological invariants (n', d') 
and (n", d") respectively, and the isomorphism is on the triple T. 

2. Define 5°+(n", d", n', d') C 5„+(n", d", n', d') to he the set of all ex- 
tensions for which moreover T is -stable. In an analogous manner, 
define S,- (n", d", n', d') and (n", d", n', d') C 4+ (n", d", n', d') . 

etc ^ 

3. Define 

S.t - U 4j (n", d", n', d') , = U ^It n', d') 
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where the union is over all {n[,n'2,d'i,d'2) and (n", 722, ci'i', ^2) such that 
the above conditions apply, and also ,"2 / < 



4- Similarly, define 

where the union is over all {n[,n2,d[,d2) and (n", n'g', rf'/, dg) such that 
the above conditions apply, and also , > „ . 

Remark 6.7. It can happen that or is empty. For instance there 
may be no possible choices of {n[, n'2, d[, d'2) and {n'(, n'a', d", ^2) which satisfy 
all the required conditions. In this case, the implication of the next lemma 
is that one or both of the flip loci S^± is empty. 

Lemma 6.8. There are maps, say : S^± — > -^a^' ''^hich map triples to 
their equivalence classes. The images contain the flip loci S^±. 

Proof. The existence of the maps is clear. The second statement, about the 
images of the maps, follows by Proposition |6.4| . Indeed, suppose that T 
represents a point in and that 

^ T' ^ T ^ T" ^ 



is an extension of the type described in proposition |6.4| , with T' a triple of 
type (n', d') and T" a triple of type (n", d"). Then (n', d') and (n', d') satisfy 
conditions ( |6.5D and ( |OD . Furthermore, since fi^+{T') < fi^+{T"), we must 

have -rr-T < . Thus T is contained in f"'"(iS°+). A similar argument 

shows that S + is contained in t>"(iS°_) □ 



6.2 Codimension Estimates 

Let € («m, «m) be a critical value for triples of type (n, d). Fix (n', d') = 
(n']^, 77-2, (i'l, ^2) and (n", d") = (n'/, 77-2, (i'/, ^2) as in Definition |6^ . For simplic- 
ity we shall denote the moduli spaces of aj-semistable triples of topological 
type (n',d'), respectively (n",d"), by 

^:±=-^.^(n''d') and Ar',=AC,±(n",d"). 
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Proposition 6.9. Ifoc ^ 2(^ — 2 then the space S^±{n", d", n', d') is a locally 
trivial fibration over M' ± x M"±, with projective fibers of dimension 



etc 

-X(n",d",n',d')-1 . 
In particular, S^± (n", d", n', d') has dimension 

1 - x(n', d', n', d') - x(n", d", n", d") - x(n", d", n', d') , 



where x(n', d', n', d') etc. are as in section [373 . 

Proof. From the defining properties of S^±{n", d", n', d') tliere is map 

4±(n",d",n',d')^Ar±xAr', (6.7) 
wliicli sends an extension 

^ T' ^ T ^ T" ^ 

to tlie pair ([T'], [T"]), wfiere [T'] denotes tlie class represented by T' and 
similarly for [T"]. We first examine the fibers of this map. 

Notice that T' and T" satisfy the hypothesis of Proposition |5.19| and 



therefore of Corollary 5.20 . Notice moreover that, since /i^±(T') < /i^±(T"), 
it is not possible to have T' = T" . Thus (cf. Corollary p.20| and Proposition 
5l8|(2)) we get 



dimP(Exti(r",T')) = dimExt^T", T') - 1 
= -x(T",T')-l 

= -x(n",d",n',d')-l, (6.8) 

which is independent of T' and T" . It remains to establish that the fibration 
( |6.7| ) is locally trivial. If the coprime conditions GCD(?t,'^, n'g, d'^ + d'^) = 1 = 
GCD(n'/, ?T,2, rf'/ + ^2) hol'i then the moduh spaces N'^± and N'^± are fine 
moduli spaces (cf. ||30[). That is, there are universal objects, say W and 
W, defined over M' ± x X and M"± x X. These can be viewed as coherent 
sheaves of algebras (cf. [^), or more precisely as examples of the Q-bundles 
considered in [^. Pulling these back to M'^± x M'^± x X we can construct 
Hom{U" ,W) (where we have abused notation for the sake of clarity). Taking 
the proiection from Af' ± x Af"± x X onto Af' ± x Af"±, we can then construct 
the first direct image sheaf. By the results in we can identify the fibers 
as hypercohomology groups which, in this case, parameterize extensions of 
triples. We thus obtain S^± as the projectivization of the first direct image 
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of Hom{U" ,W). If the coprime conditions fail, then the universal objects do 
not exist globally. However they still exist locally over (analytic) open sets 
in the stable locus in the base f/' ± x JV"±. This is sufficient for our purpose 

etc Oc 

since by construction the image of the map in ( |6.7| ) lies in the stable locus. 
The result now follows from (|6.8| ) and formula ( ^.16| ) (in Theorem |5.21|) as 



apphed to Af'± and Af"±. □ 

Corollary 6.10. Ifac > 2g — 2 then the lociS^± C are locally contained 
in subvarieties of codimension hounded below by 

min{-x(n",d",n',d')} , 

where the minimum is over all (n', d') and (n",d") which satisfy and 



<H) '^^d also ^ < (m the case of S^+) or ^ > (m the 

case of S^-). The same is true for 5^+ when ac = 2g — 2. 

Proof. If ac > 2g — 2 then we can assume af ^ 2g ~ 2. Clearly also, 
^ 2g — 2 when = 2g — 2. Thus by Theorem p.21| the moduli spaces 
^± are smooth and have dimension 1 — x(ni, n2,di,d2)- By Corollary ^.16| 
and Proposition we get 



dimjV^i = 1 - xin'i, d[, n'^, 4) - xK', c^i, 4') 

- xK, d[,nl, 4') - X«> d'i, 4, 4) 

=dim5„± (n", d", n', d') - x«, d'[, 4, 4). 



□ 



In order to complete the estimate on the size of the flip loci we thus need 
to estimate the expressions xiT" , T') = x(n", d", n', d'). The basic idea is to 
identify x{T', T") as a hypercohomology Euler characteristic for the complex 
C*{T", T') defined in ^.12 and to notice that the complex is itself a holomor- 



phic triple. As such it ought to satisfy a stability condition induced from the 
stability condition of T' and T" . The right way to obtain the stability con- 
dition for C*{T",T') should be via the Hitchin-Kobayashi correspondence, 
in a way analogous to the case of vector bundles: to show that the tensor 
product of two stable vector bundles is semistable, one constructs a fiat uni- 
tary connection on the tensor product from fiat unitary connections on each 
of the factors. In the case of holomorphic triples one considers solutions to 
the so-called coupled vortex equations instead of fiat connections. However, 
it turns out that one cannot construct (at least in a simple way) a solution 
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to the coupled vortex equations for C*{T",T') from solutions on T' and T". 
Instead one needs to consider a so-called holomorphic chain. These objects 
were studied in and generalize holomorphic triples. Next we recall the 
relevant results and definitions. 



6.3 Holomorphic chains 

A holomorphic chain is a diagram 

r TP 'f'"^ TP fPrn-l <t>l 

■ J^m ^ ^ ■ ■ ■ ^ ^0, 

where each Ei is a holomorphic bundle and (pi: Ei -Ej-i is a holomorphic 
map. Let 



m j } 



fi{C) = fi{Eo ®---®E, 
rk(Ei) 

Xi{C) = =^ — TT^' « = 0, . . . ,m. 
For (X = (ai, . . . , a^) ^ I^™, the aslope of C is defined to be 

m 
i=l 

The notions of a- (semi) stability and a-polystability are then defined via 
the standard a-slope condition on sub-chains. Note that holomorphic chains 
generalize holomorphic triples: a holomorphic triple is a holomorphic chain of 
length 2, and the stability notions coincide, taking o: = {a). There are natu- 
ral gauge theoretic equations for holomorphic chains, which we now describe. 
Define r = (tq, . . . , r„) G M"+^ by 

Ti = ^a{C)-ai, i = 0,...,m, (6.9) 

where we make the convention ao = 0. Then o: can be recovered from r by 

ai = To-Ti, z = 0, ...,m. (6.10) 

The T-vortex equations or chain vortex equations 

yf-[AF{Ei) + - = n Idij^, i = 0, . . . , m, 

are equations for Hermitian metrics on E'o, . . . , Em- Here F{Ei) is the cur- 
vature of the Hermitian connection on Ei, A is contraction with the Kahler 
form and vol(X) = 27r. By convention 0j = for i ^ 1, . . . ,m. With these 
preliminaries we can now state the Hitchin-Kobayashi correspondence for 
holomorphic chains. 
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Theorem 6.11 {^)- A holomorphic chain C is cx.-polystable if and only if 
the T -vortex equations have a solution, where ol and r are related by (|6.9|) . 

Remark 6.12. To be consistent with the previous notation for holomorphic 
triples we use here a slightly different notation from that of [Q: in that 
paper a parameter vector a. = (ao, «i, • • • , ctm) is used; in these terms our ex. 
is given by ck = (ckq — cti, ■ ■ ■ , c^o — «m)- 

Consider the length 3 holomorphic chain 

C-{T", T') : E'r ®E'^^ E'l* (E)E[(B E'^ ® E'^ ^ E'^ (g) E[, (6.11) 
where 

a2(V^) = (0V,-V'0"), 

We shall sometimes write this chain briefly as 

C-{T",T'): C2^Ci ^Cq. 

Note that the last two terms of C*(T",T') is just the complex C'iT",T'). 
Note also that C*{T", T') is not in general a complex. Our goal in this section 
is to prove, using the Hitchin-Kobjyashi correspondence for chains, that if 
T' and T" are a-polystable then C*{T",T') is a-polystable for a suitable 
choice of a. 

Lemma 6.13. Let T' and T" he holomorphic triples and suppose we have 
solutions to the {t[,T2) -vortex equations on T' and the {t[' ,T2) -vortex equa- 
tions on T" , such that t[ — r" = T2 — tI^. Then the induced Hermitian metric 
on C*{T",T') satisfies the chain vortex equations 

foldco, (6.12) 
Tildci, (6.13) 
fsldc,. (6.14) 

for T = (fo, fi, fs) given by 

^ — t' t" 
^0 — T^l — ^2 ' 

~ I II I It 

Ti = T^-T^ =r2-T2, 

I II 
T2 = T2-T^. 



V^AE{Co) + aifli* 
-lAF(Ci) + 0202* — ai*ai 
^/^AE(C2) - a2*a2 
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Proof. We shall only show that the induced Hermitian metric satisfies ( |6.13D , 
since the proofs that it satisfies the two remaining equations are similar (but 
simpler) . 

The coupled vortex equations for T' and T" are 
^/^AF{E[) + <p'r = r^Id^j, ^/^AF{E'I) + <P"<P"* = t'^Ue'^, 



We shall write the left hand side of (|6.13|) in terms of these known data of 
the triples T' and T". First, we note that 

FiE';) = -FiEl)\ 2 = 1,2, 

and similarly for F{E"*). Hence 

F{Ci) = F{E'I* ®E[® E'f ® E'^) 

= {F{E';*)(^ld + M^F{E[), ®Id + Id®F(E^)) 

= {-F{E';y ^ld + ld^F{E[), -F(E^')*®M + Id®F(E^)). (6.15) 

Next we calculate a^: note that for ^ ® x G Co and {rji ®yi,ri2® 2/2) ^ Ci we 
have 

(a*(^®x), (?7i ® 1/1, ?72 01/2) >c.^ 
= {i ® X, aiirii (g) yi, r]2 ® 2/2))^.^ 

= -{4>"'*{0,Vi)e'^' {^^yi)E[ + (^'^2>^,,* (0'*(x),y2>^, 

= ((-0"**(O ® a;, e ® 0'*(x)), (r/1 ® yi, r/2 ® l/2)>c^. 

Hence, 

a\{i®x) = (-0"**(O®x, e®0'*(a;)). (6.16) 
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Similarly, to calculate consider S,®x E C2 and {rj\®y\, 772 ® 2/2) G C*!, then 
{alirii (g) yi, 772 ® 2/2), ^ ® a;)^^ 
= ((^1 ® 2/1,^2 02/2), 02(^02;))^^ 
= ((??i ® 2/1, ® 2/2), ® </>'(x), ® a;)>c, 



= (r7i®2/i,e®0'(a;)>^»*^^, + (r72®2/2,-0"*(O®a;)^„.^^, 



Thus, 

a2(m ® 2/1,^2 ® 2/2) = ^71 ® 0'*(2/i) - 0"**(^72) ® 2/2- (6.17) 
Using ( p.lTp and ( |6.16| ) we can now calculate for (r/i ®yi,rj2® 2/2) G Ci. 

0202 (r/i ® 2/1,^72 ® 2/2) 

= (r/i ® 0'0'*(2/i) -0"**(r72) ® 0'(2/2), -0"*(r7i) ® 0'*(2/i) +0"V"**(r/2) ^2/2) , 

(6.18) 

and 

atai('7i ®yi,ri2® 2/2) 

= (0"*>"*(r7i)®2/i-0"**(r?2)®0'(2/2), -0"*(r]i)®r(2/i) + ^2®0'>'(2/2)). 

(6.19) 

Putting together ( p.l5|) , ( |6.18| ) and ( |6.19| ) we finally obtain 

[\^KF{Ci) + 0202* - aiai) {t]i ®yi,ri2® 2/2) 
= (r^i ® (v^AF(EO + 0V*)(2/i) + (-v^AF(i?n* + (^0 ® 

r^2 ® (v^AF(i?^) - r<P') (2/2) - (v^AF(E^')* - <P'U"'*) im) ® 2/2) • 

(6.20) 

Notice that the unpleasant mixed term (—</>"** (^72)®^' (2/2)5 ~4>"^ {Vi)®4>'* {Vi)) 
appears both in a\ai and 0203 and therefore cancels. This would not have 
been the case if we had considered the vortex equations on the triple C'(T", T') 
and is the reason why we must consider the chain C'{T" ,T'). Combining 
(16.201 ) with the coupled vortex equations (or their transposes) for the triples 
T' and T" we get 



{v^AF{Ci) + 0202* - ai*ai) {t]i ®yi,ri2® 2/2) 

= ((^{-0^7i® 2/1,(^2 -0^2® 2/2). (6.21) 
Since t[ — = t!^ — t!^ this concludes the proof. □ 
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Proposition 6J^4. Let T' and T" he a-polystable triples. Then the holo- 
morphic chain C*{T",T') is cx-polystahle for ex = (0:1,02) = (a, 2a). 

Proof. Since T' and T" are a-polystable, it follows from the Hitcliin-Koba- 
yaslii correspondence for triples that they support solutions to the {t[,T2)- 
and {t[, r2)-vortex equations, respectively, where a = t{— r2 = t'{—T2. Notice 



that t[ — t'I = T2 — Thus Lemma |6.13| shows that C*{T",T') supports 
a solutions to the chain vortex equations for r 



Now the Hitchin-Kobayashi correspondence for chains (Theorem |6.11|) and 
(loop imply that C'{T",T') is a-polystable for 



ai 

^2 



// 1,11 

T"2 - ^2 + ^2 



a, 



2a. 



□ 



The following Lemmas will be needed in the next section: 



Lemma 6.15. LetV = (E;,E^,0') and T" = {E'{ , E'^ , 4>") be triples for 
which the chain C*{T",T') is ex. = {a,2a)-poly-stable. Let 



Ci = E'C ®E[®E'^* ®E'2, 
Co = E'f®E[, 



and ai : Ci Co be defined as in ( \6.1J\) . Then the following inequalities 
hold. 

deg(ker(ai)) ^ rk(ker(ai))(/i„(T') - /ia(T")), (6.22) 

deg(im(ai)) ^ (rk(Co) - rk(im(ai))) (/i,(T") - /i„(T') - a) + deg(Co). 

(6.23) 



Proof. If rk(ker(ai)) = then ( |6.22[ ) is obvious. Assume therefore that 
rk(ker(ai) > 0. Using ker(ai), we can then define a quotient of the chain 
C*{T",T') by 

/C: ^ker(ai) ^ 0. 
Thus, since HcxiJO) = /i(ker(ai)) + a, it follows that 

/i(ker(ai)) + a ^ ^„(C^(T", T')) = /ia(T') - fiaiT") + a. 
We therefore have 

/x(ker(ai)) ^/i,(T')-/ia(T"), 
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which is equivalent to ( |6.22| ). The second inequahty, i.e. ( |6.23| ), is obvious if 
rk(im(ai) = rk(Co). We thus assume rk(im(ai) < rk(Co). Using the cokernel 
coker(ai) (or its saturation if it is not torsion free), we can define a sub-chain 
of the chain C>(r",T') by 

Q: ^ coker(ai). 

By the a-poly-stabihty of C*(T", T') we have fi^iQ) > iioc{C*{T\ T')). This, 
together with the fact that 

( A ( \ ^ deg((:7o) - deg(im(ai)) 
^ rk(Co)-rk(im(aO) ' 

leads directly to |6.23| . □ 



Lemma 6.16. Let 

— >T' — >T — >T" — ^ 

he an extension of a-poly stable triples, where T' and T" are non-zero and 
f^a{T') = fia{T") = Hair). Let a^n and um be the extreme a values for the 
triple T , as defined in (^.3^ ) and ( |5.^[ j, with the convention that um = oo if 



Ui = n2 in T. Then the map 

ai : E'l* ®E[(B E'f ® E'^ ^ E'f ® E[ 
is not an isomorphism if am < « < ctAf- 



Proof. If ai is an isomorphism then, applying Lemma [4. 12| fibrewise, it follows 
that one of the following alternatives must occur: 

(a) E[ = E'^ = and 0' = 0" = 0. 

(b) E'{ = 0, E[, E'^, E'i ^ and cj)' : E'^ ^ E[. 

(c) E'^ = 0, E'l, E'J, ^ and 0" : E'^ E'l 

We shall consider each case in turn. Case (a). In this case T' = (0, E2, 0), T" = 
(El, 0, 0), T = (El, E2, 0), and the extension ^ T' ^ T ^ T" ^ is triv- 
ial. It follows from fia(T') = fiaiT) that a = fi{Ei) — fi{E2) = Case (b). 
In this case we have Ui = n[ and n2 = n'^ + = n[ + n'g. Hence n2 > ni. 
Furthermore, from /x„(T') = /i„(T) we get /i(Ei) + f = /i(Ei © E2) + 
i.e. a = ^^"^^ cum = ctM- Case (c). In this case we have n2 = ^2 and 
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rii = n[ + n'l = n[ + n'g. Hence rii > n2. Furthermore, from fia{T") = fia{T) 
we get a = am = Q^m- If = ^2 then case (a) is the only possibihty, 
so a = am- If ni ^ n2, then (a) or exactly one of (b) and (c) are the only 
possibilities, depending on whether ni < n2 or rii > n2. In both cases we see 
that a = am or a = aM- □ 



6.4 Estimate of x{'^"^ d", n', d') and comparison of mod- 
uli spaces 

Let ac G {am,aM) be a critical value with ac ^ 2g — 2, and let af be as 
in (pD. Fix (n',d') = K,n^,d'i,4) and (n",d") = {n'l,n'^, d'l, d'^ such 
that equations ( |6.5| ) and ( |6.6| ) are satisfied, and such that ^7^^ 7^ ^„'^^„ . 
Let T' and T" be triples of type (n',d') = {n[, 7X2, d[, and (n", d") = 
(n", 71-2, ci'/, ^2) respectively. Suppose that T' and T" satisfy: 

• both are Oc-semistable with fiadT') = fiadT"), 

• if -TT-r < -rrr-FT then T' and T" are both a+-stable, 

• if , > ,,"1^ „ then T' and T" are both a7-stable. 

It follows by Proposition |6.14| that the chain C'{T", T') is (aj, 2a;i")-polystable 
(if , < —rrr-n) or it is (a7, 2Q;~)-polystable ( if , < „ ). 

Lemma 6.17. T/ie chain C*{T",T') is {ac,2ac) polystable. 

Proof. The critical values for the chain form a discrete set of points in the 
{ai,a2) plane. We can thus pick e > so that, with af = ac i: e, the 
point (a^,2a^) is not a critical point. We can in fact assume that there 
are no critical points in B°{ac,2ac), i.e. in the punctured ball of radius e 
centered at {ac,2ac)- Thus {af,2af) polystability is equivalent to 2ac) 
polyst ability. □ 



Proposition 6.18. LetT' = {E[, E!^, (j)') andT" = {E'l , E'^ , (p") be triples as 
above. In particular, suppose that both are ac-semstable and that /ia^(T') = 
Ha,{T"). Then 

x{T",T')^l-9<0 
if one of the following holds: 
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1. Oc > 2g — 2 and T' , T" are both are -stable, 

2. 2g — 2 and T' , T" are both are -stable. 

Proof. From the long exact sequence ( |5.13| ) and the Riemann-Roch formula 
we obtain 

X(T", T) = {l-g) (rk(Ci) - rk(Co)) + deg(Ci) - deg(Co). (6.24) 



where Ci and Co are as in (|6.11|) . From Proposition 5.14| , we know that 



C'(T", T') is (aj, 2Q;+)-poly-stable or (a^ , 2a^ )-poly-stable. By the Lemma 
5.17| it follows that C'{T", T') is also a = {ac, 2a;c)-poly-stable. Thus Lemma 



5.15| applies, and we can use the estimates ( |6.22|) and (|6.23| ). Together with 



deg(Ci) = deg(ker(ai)) + deg(im(ai)) , (6.25) 
rk(Ci) = rk(ker(ai)) + rk(im(ai)) , (6.26) 



these yield 



deg(Ci) ^ (/x,,(T') -/i..(T"))(rk(Ci) -rk(Co)) 

- ac(rk(Co) - rk(im(ai))) + deg(Co 

Using that /iQ,^(T') = fia^(T"), we can then deduce that 

deg(Ci) - deg(Co) ^ -ae(rk(Co) - rk(im(ai))). 
Combining this with (|6.24| ) we get 



X(T",T') ^ (1 - g){THCi) - rk(Co)) - ac(rk(Co) - rk(im(ai))) . (6.27) 

If ^ 2(7 — 2 then we get 

xiT"X) ^ (l-^7)(rk(Co)+rk(Ci)-2rk(im(ai))), 

with equality if and only if = 2g — 2. Furthermore rk(ai) ^ rk(Co) and 
rk(ai) ^ rk(Ci), with equality in both if and only if oi is an isomorphism. 
Thus in all cases we get xiT" , T') ^ 0, with equality if and only ii ac = 2g — 2 
and ai is an isomorphism. But by Lemma |6.16| if Um < oic < olm then a\ 
cannot be an isomorphism. Thus in all cases we get rk(Co) + rk(Ci) — 
2rk(im(ai) ^ 1) and hence x(^",^') ^ 1 - ^- □ 



Combining Proposition |6.18| with Corollary |6.1U| , we obtain 
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Theorem 6.19. Let Oc G {am,C(M) be a critical value for triples of type 
(n, d). If > 2g — 2 then the loci S^± C J^^± o,re contained in suhvarieties 
of codimension at least g — 1. In particular, they are contained in suhvarieties 
of strictly positive codimension if g ^ 2. If ac = 2g — 2 then the same is true 

Corollary 6.20. Let ai and he any two values in {am,(yM) such that 
am < «! < ^2 < «M cind 2g — 2 ^ ai. Then 

• The moduli spaces Af^^ and Af^^ have the same numher of connected 
components, and 

• The moduli space A/"^^ is irreducible if and only if M^^ is. 



Proof. This follows immediately from Theorem |6.19| if ai and a2 are non- 
critical, and from Theorem |6.19| together with Lemma if either of them 
is critical. □ 



7 Special values of a 

Throughout this section we will assume that the triple {Ei,E2,4>) has type 
(ni, 722, (ii, ^2), with ni ^ n2- The case ni < n2 can be dealt with via 
duality of triples. In this section we identify some critical values in the range 
{am, o;m) and describe their significance for the structure of a-stable triples. 

7.1 The kernel of and the parameter a 

Definition 7.1. For each integer ^ j < n2 set 

2nin2 . . , 

n2[ni - n2) + [J + l){ni + n2) 



Proposition 7.2. Let T = {Ei,E2,4>) be a triple in which ni ^ ^2. Let 
N (Z E2 be the kernel of (p : E2 — > Ei. Suppose that T is a-semistable for 
some a > aj. Then N has rank at most j . In particular, ifT is a-semistable 
for some a > ao then N = 0, i.e. is injective. 

Proof. Suppose that 

rk{N) =k>0 . (7.2) 
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We consider the subtriples T/v = (0,A^, 0) and T/ = {I,E2,4>), where / de- 
notes the image sheaf im(0). If 7^ 0, then the triple T^y is a proper 
subtriple, and so is Tj since rii ^ n2. The a-semistabihty condition apphed 
to T/v yields 

/iAT + a ^ /i + 



Til + n2 

where /i/v denotes the slope of and /i is the slope of Ei®E2. Rearranging, 
we get 

aril ^ (ni + n2)(yU - yUAr) . (7.3) 
The a-semistability condition applied to T/ yields 

/i(E2©/) ^/i + a^^ , (7.4) 
% + n2 ni+n2 

where i = rk(J). Furthermore, from the exact sequence 

— >N — >E2^I — >0 , (7.5) 



we get 



k + i = n2 , (7.6) 
k^iN + = ^2^2 ■ (7.7) 



Using ( [7.7| ) we can write 



= /^^ 7.8 

/n2 — 

and hence ( [7. 41) yields 

-/c(ni + n2)/iiv ^ (?T-i + ?^2)((2ri2 - /c)/i - 2n2H2) + an2{n2 - k - Ui) . 

(7.9) 

Combining k times ( [7.3| ) and ( |7.9| ) yields 

2^2(^1+^2) , , 

77-2(^1 - ^2) + k[ni + 77-2) 

(/ii-/i2). (7.10) 



n2(ni - ^2) + k{ni + n2) 
We have thus shown that if rk(A^) = k and the triple is a-stable, then 

a ^ ak-i 



70 



where ak-i is given by ( [7. 1| ) with j = k — 1. Equivalently, if the triple is 
a-semistable for some a > Ok-i, then rk(A^) 7^ k. But 

We can thus conclude that if the triple is a-semistable with a > ak-i, then 
the rank of is strictly less than k. In particular, if a > oq, where 

«o = — 7 '^'^I'f^'i (7.11) 

then T is injective. □ 
We thus have the following. 

Corollary 7.3. Let a > Oq, where is given by l \7.11\ ). 



(1) An a-semistahle triple {Ei,E2,4>) defines a sequence of the form 

— >E2^Ei — >F®S — ^0, (7.12) 
where F is locally free and S is a torsion sheaf. 

(2) If ni = n2 then an a-semistahle triple {Ei,E2,(f)) defines a sequence of 
the form 

— >E2^Ei — >S — ^ 0, (7.13) 
where S is a torsion sheaf of degree di — 62- 

Lemma 7.4. Let he given hy (\7.1J\) . 

(1) If rii > n2 then 

^2(^1-712) 2nin2 

"0 = — 7 : (Xm = — 7 7— : ttm, (7.14) 

n2{ni - n2) + ni + n2 n2{ni - ^2) + ni + n2 



where a.^ and are given hy ( |5.5[ ) and ( |5.^[ ), respectively. 

(2) If ni = n2 = n then 

ao = nam = ^il^i — 1^2) = di — ^2. (7-15) 

(3) Ifui ^ n2 then ao ^ am, with equality if and only if am = or n2 = 1. 

Proof. Parts (1) and (2) are immediate. Using (1) we compute 

ao - "m = — 7 7— \ \n2 - V)am , 

n2{ni - n2) + ni + ^2 

from which (3) follows. □ 
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7.2 The cokernel of and the parameter a. 



In this section we will assume that > n2. The range for a is then [a.^, cxm]-, 
where and are given by ( |5.tj| ) and (|5.3|). Let us define 

tti := ctM 7 r- (7.16) 

n2(ni - na) 



Proposition 7.5. Suppose that a triple T = {Ei,E2,4>) of the form \7.12^ 
with rii > 77,2 is a-semistahle for some a > am- Then 

n2{ni -n2) , . 

s < — ■ r-(aM - a) , 

('^i + ^2) 

where s is the degree of S. In particular, if a > at, then S = 0. 

Proof. If T = {Ei,E2,4>) is of the form in ( |7.12| ), with 5 7^ 0, then we can 
find a proper subtriple T' = {E[, E2, 0) of the form 

— >E2^E[ — > S — >0 , (7.17) 

Indeed, E[ is the kernel of the sheaf map Ei — > F (B S — > S. Notice that 
n[ = n2 and d[ = d2 + s, where n[ , d[ denote the rank and degree of E[, 
etc. We compute 

A„(r) = ^(,.-,0 + f(^)+^. (T.18) 

Til + ^2 2 V rii + ^2 / 2n2 



But 



ni aM ni-n2 

-(/ii -/i2) = — 



ni+n2 2 \ni + n2 

and hence 

^a(T) = - — r[a-aM + —, . (7.19) 

2(ni + 712) V n2{ni - ^2) / 

If the triple is a-semistable then Aa(T') ^ and the result follows. □ 
Let us define 

cte = maxjam, cto; (7.20) 
The following is then immediate. 
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Proposition 7.6. Let a > ae- An a-semistable triple {Ei,E2,(p) defines an 
extension 



— >E2^Ei — >F — . 0, (7.21) 

with F locally free. 

It turns out that for extension hke ( |7.21| ), arising from semistable triples 
the dimension of H^[E2 ® F*) does not depend on the given triple. More 
precisely: 

Proposition 7.7. Let (E'l, £'2, </>) be an a-semistable triple defining an ex- 
tension like l \7.21\) . Then H°{E2 ® F*) = and hence 

dimH\E2 O F*) = ria^i - rii^a + n^im - n2){g - 1). (7.22) 

Proof. From Lemma 4.5] we have that the a-semistability of {Ei,E2,(f)) 
for arbitrary a implies that H^{Ei (8> E2) = 0. From (|7.21| ), we have an 
injective homomorphism F* E^, which after tensoring with E2 gives that 
H^{E2 ® F*) injects in H^{Ei (E) E2), and hence the desired vanishing. By 
Riemann-Roch we obtain ( [r.22|) . □ 



8 Moduli space of triples with ni ^ n2 

Throughout this section we assume that ni > n2- As usual, the case rii < n2 
can be dealt with by triples duality. Recall that the allowed range for the 
stability parameter is am ^ a ^ c^m, where am = /ii— /i2 and aM = j^zi^c^m, 
and we assume that fii — ^2 > 0. 

We are now ready to jump all the way up to the large extreme value of 
the range, and describe the moduh space for aM- 



8.1 Moduli space for a = aM 

Proposition 8.1. Let T = {Ei,E2,(f)) be an aM-polystable triple then Ei = 
im (j)Q)F, and T decomposes as the direct sum of two aM-polystable triples of 
the same aM-slope, T' and T" , where T' = (im0, i?2,0)j O'lT'd T" = (F, 0,0). 
In particular, T is never aM-stable. Moreover, E2 = im0 and E2 and F are 
polystable. 
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Proof. By Proposition, |7.6| T defines an extension 

— >E2^Ei — >F — ^ 0, (8.1) 

with F locally free. Let T' = (im0, £^2, 0)- Of course : ii^2 ^ ini0 is an 
isomorphism, and 

but this is equal to fiaM^'^) hence T cannot be ctAf-stable and must de- 
compose as T' © T", where T" = {F, 0, 0). It is clear from the polystability 
of T that T' and T" are aAf-polystable with the same aAf-slope. Apply- 
ing the aM-semistability condition to the subtriples {E'2, (j){E2) , (p) C T' and 
{F', 0, 0) C T", we obtain that /i(E^) ^ fx{E2) and fi{F') ^ f/{F), and hence 
E2 and F are semistable. In fact the polystability of T' and T" imply the 
polystability of E2 and F. □ 

Using Proposition |9.1| and Corollary ^]2| from Section p.l| , we obtain the 
following corollary of Proposition |8.1| : 

Corollary 8.2. Suppose that rii > n2 and — ^2 > 0. Then 

A/'«A,(ni,n2,c/i,(i2) = J^aMiiT'2,n2,d2,d2) X M{ni - n2,di - ^2) 
^ M(n2, rfs) X M{m - n2, c/i - ^2) 

where M{n, d) denotes the moduli space of polystable bundles of rank n and 
degree d. In particular, A/'a^,^(ni, n2, (ii, ^2) is irreducible. 



8.2 Moduli space for large a 

Let ai be the largest critical value in (am,«M), and let Ml (respectively 
Ml) denote the moduh space of a-polystable (respectively a-stable) triples 
for ai < a < aM- We refer to Ml as the 'large a' moduli space. 

If a triple T = {Ei,E2,(f)) defines an extension of the form ( |7.21| ), then 



J = imc/) is a subbundle with torsion free quotient in Ei, and (j) : E2 — ^ I 
is an isomorphism. Thus we get a subtriple Tj = (J, i?2,0) in which the 
bundles have the same rank and degree, and is an isomorphism. 

Proposition 8.3. Let T = {Ei,E2,4>) represent a point in Ml, i.e. suppose 
that the triple is a-semistable for some a in the range aL < a < c^m ■ Then 

(1) the triple Ti = (/,£'2,0) is au-semistable 
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(2) the bundle E2 is semistable 

Proof. (1). Let T' = {E[, E'2, (p') be any subtriple of T/. Since T' is also a 
subtriple of T, we get 

^^a{T') ^ fiaiT) . (8.3) 
A direct computation shows that 

ni+n2 2 

rii — 77-2 / 

2(ni + n2) 

where in the last line we have used the fact that ni > n2 in T and hence 
o^M = ni-n2 ~ ^2) = 2(/i(-F) — /U.2). Thus for all < a < we have 

Ha{T') - fia{Ti) ^ ^rp—^{aM - a) . 
2[ni + n2> 

Taking the limit a — aAfj we get 

fia^AT') - HaM^Ti) ^ , 

i.e. Tj is aAf -semistable. 

(2). Let E2 C E2 be any proper subsheaf. Then {cfy^E'^) ^ E'2, (p) is a 
subtriple of T/. Since : -^2 — ^ 4>{E2) is an isomorphism, this subtriple has 
fi{(j){E'2) = fi{E2) and n'g = n[. The ctAz-semistability condition of T/ thus 
gives 

/^(^2) + — ^ + — , 

(where we have made use of the fact that fi{(j){E2) = fJ'{E2) = ^2)- It follows 
from this that //(-Eg) ^ /i2, i.e. that E2 is semistable. □ 

Proposition 8.4. Suppose that the triple T = (i?i, i?2, is of the form in 

— > E2^ El — y F — ^ 0, 

with F locally free. Then there is an e > such that F is semistable if the 
triple is a-semistable for any a > um — e- Indeed the conclusion holds for 
any 

< e < , ^ , (8.5) 
where m = rii — n2 = rk(F). 
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Proof. Let F' G F he any proper subsheaf. Denote the rank and slope of F 
(resp. F') by m and ftp (resp. m' and fip')- We can always find E[ C i?i 
such that F' = E[/E2, i.e. such that we have 

— > E[ — > F' — ^ 0. 

Let T' = {E[, E2,4>). For convenience, define 

A, = A,(T') = fi^iT) - fi^{T) . (8.6) 



Using 



5 



Hi = n2 + m 
n'l = n2 + m' 
riifii = n2/i2 + mfip , (8.7) 
n[ij[ = n2/i2 + m'nF' , 



we get 



{2n2 + m){2n2 + m') fm-m'\ 
- /i^ = A. - [-^ J (a - 2{^^, - /.2)) . 

(8.8) 

But 2{hf — /i2) = OiM- Thus, setting 

a = — e , (8.9) 

we get 

{2n2 + m){2n2 + m') ( m - m' \ e 

-^' = 2^' + ) 2 ■ ^'-''^ 

If now we take 

e 1 

- < 



2 mim — IV ' 



then for all < m' < m we get 



m — m'\ e 1 , , 

:r < — TT . 8.11 



m' / 2 m(m — 1) 
Hence, if the triple is a-semistable, so that Aq, ^ 0, then we get 

1 

HF' - fJ'F < 7T • (8.12) 

m[m — 1) 

Since /If and ^f' are rational numbers, the first with denominator m, and 
the second with denominator m' ^ (m — 1), equation ( p.l2|) equivalent to the 
condition /x^?/ — /i^^O. □ 
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We can combine Propositions \l.Q[ f>A\ and p.3| to obtain the following 



Proposition 8.5. Let T = {Ei,E2,4>) be an a-semistahle triple for some a 
in the range ul < a < um ■ Then T is of the form 

— > El — > F — ^ 0, 

with F locally free, and E2 and F are semistable. 

In the converse direction we have: 
Proposition 8.6. Let T = [Ei, E2, (p) be a triple of the form 

— > E2^ El — > F — ^ 0, 

with F locally free. If E2 is semistable and F is stable then T is a-stable for 
a = ttM — € in the range ai < a < aM- 

Proof. Any subtriple T' = {E[,E2,(f)') defines a commutative diagram 
> E2 — ^ El > F > 



> E'2 E[ > F' > 0, 

where F' G F. Then 
A, = A„(T')=;U,(T')-^a(T) 

= ^{E[®E'2)- ^l{El®E2) + a{^^^ ^). (8.13) 

Denote the rank and slope of F (resp. F') by m and /xj? (resp. m' and i^f')- 
Using 

rii = n2 + m ^ 
Ui = n2 + m , 
rii/ii = n2fJ,2 + mfip , 
n[fj.[ = ?T,2/i2 + m'fxp' , 
and the fact that om = 2(/iir — (12), and setting a = q;m — we obtain 

+ 1^2 + mfip/ 2n2 + fX2 + m^iF ^, m "^2 ^2 

'^a - ^^-r- ; 7, \ ^ ^\i^F - At2jl- 



2r^2 + 2n2 + m 2n'2 + m' 2n2 + m ' 

n'2 n2 ^ 



2^2 + vn! 2n2 + m 
2n'2 / I \ m' riom — n2m' 

(8.14) 
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Clearing denominators in ( ^.14| ) we obtain 



A. = n'2(2A2 - T^^^e) + m'(Ap + ;-^^6), (8.15) 

where 

Aq, = (2n2 + m') Aq,, A2 = ^2 — fJ'2, and Ap = fxpi — fip. (8.16) 

Now suppose that E2 is semistable and F is stable. The semistability of E2 
implies that 

m 

2A2 - ■ e < 0, 

2n2 + m 

while the stability of F implies there exists 6 > such that 

< 0. 

Thus by taking 

2^2 + m ^ 

e < 6, 

n2 

we have 

2^2 + m 

and hence Aq < 0, completing the proof. □ 

Theorem 8.7. Letrii > n2 and di/rii > d2/n2. The moduli space J\fl{ni,n2,di, 
is smooth of dimension 

{g - l){nl + n\- nin2) - nid2 + ^2^1 + 1, 

and is birationally equivalent to a -fibration over M^{ni — n2,di — ^2) x 
M''(n2, (^2); where 

N = n2di - nid2 + ni{ni - n2){g - 1) - 1. 

In particular, N'[{ni,n2,di,d2) is non-empty and irreducible. 

If GCD{ni — n2,di — ^2) = 1 and GCD{n2,d2) = 1, the birational equiv- 
alence is an isomorphism. 

Moreover, A/L(ni, n2, c/i, ^2) is irreducible and hence birationally equiva- 
lent to J\f[{ni,n2, di, d2). 



78 



Proof. For every triple T = {Ei,E2,(f)) in A/'|(ni, ?t,2, rfi, ^2), the homomor- 
phism (J) is injective and hence, by (5) in Proposition f).21[ T defines a smooth 
point in the moduh space, whose dimension is then given by (4) in Proposi- 



tion 5.4 



Given F G M**(?t,i — n2,di — ^2) and E2 G M^(n2,d2), we know from 



Proposition that every extension 

— > E2^ El — > F — ^ 0, 

determines a triple T = {Ei, E2, 4>) in Afl{ni,n2, di, (^2). These extensions are 
classified by H'^{E2®F*). In fact two classes defining the same element in the 
projectivatization FH^{E2 ® F*) define equivalent extensions and therefore 
equivalent triples. Now, 

deg(£'2 ® F*) = [ni - n2)d2 - n2{di - 6/2) = nin2{^2 - /Wi) < 

and, since E2 ® F* is semistable, then H^{E2 ® F*) = 0. Hence, by the 
Riemann-Roch theorem 

h^{E2 ® F*) = n2di - nic/2 + ni{ni - n2){g - 1). 

In particular this dimension is constant as F and E2 vary in the corresponding 
moduli spaces. 

We can describe this globally in terms of Picard sheaves. To do that we 
consider first the case in which GCD(ni — n2, ^1 — ^2) = 1 and GCD(n2, ^2) = 
1. In this situation there exist universal bundles F and E2 over X x M{ni — 
n2, di — ^2) and X x M(n2, ^2), respectively. Consider the canonical projec- 
tions TT : XM{ni—n2, di — d2) xM{n2, d2)'x M{ni—n2, (ii — ^2) xM(n2, (^2) 
u : X X M{ni — ^2, di — ^2) x M{n2, d2) ^ X x M{ni — ^2, di — ^2), and 
7r2 : X X M(ni — ^2, ^1 — ^2) x M(n2, 0^2) X x M{n2, ^2)- The Picard sheaf 

S ■= B}tx^{txI'K2 ® u*¥*), 

is then locally free (a Picard bundle) and we can identify A/L(ni, n2, c?i, ^2) = 
J\fl{ni,n2,di,d2) with V = F{S). This is indeed a fibration with = 
n2di — 721^2 + ^1(^1 — n2){g — 1) — 1, which in particular is non-empty since 
M{ni — n2, di — ^2) and M{n2, ^2) are non-empty and N > 0. 

If GCD(ni — n2,di — 6/2) 7^ 1 and GCD(n2,(i2) 7^ 1, the universal bun- 
dles and hence the Picard bundle do not exist but the projectivization over 
M^{ni — n2,di — ^2) x M'^{n2,d2) does. One way to show this is to work 
in the open set R of the Quot scheme corresponding to stable bundles. 
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The point is that an appropriate hnear group GL acts on R, with the cen- 
tre acting trivially and such that PGL acts freely with the quotient being 
M^(ni — 71,2, di — ^2) X M''^{n2, ^2). For the action on the projective bundle 
associated to the universal bundle over R, the centre of GL still acts trivially, 
and one can use standard descent arguments to obtain a fibration V over 
M'im - n2, di - 4) X M'{n2, ^2). 

W now show that Afl{ni,n2,di,d2) — V has strict positive codimension 
in Afl{ni,n2, di, ^2)- This follows from two facts. The first one is that any 
family of semistable bundles of rank rii — n2 and degree di — d2 depends on a 
number of parameters strictly less than the dimension of M*(ni — n2, (ii — (^2) 
(similarly for any family of semistable bundles of rank 77,2 and degree 0^2) ■ The 
second fact is that the dimension of H^{E2 F*) is fixed by the Riemann- 
Roch theorem (we use here that E2 and F are semistable). 

To prove the last statement, i.e. to extend the results to A/l(?t.i, 77-2, di, ^2), 
we consider the family V of equivalence classes of extensions 

— >E2^Ei — >F — ^ 0, 

where F and E2 are semistable. Clearly, V contains the family V. The family 
V is irreducible. This is because since F and E2 are semistable they vary 
(for fixed ranks and degrees) in irreducible families J-" and S2, respectively, 
and as shown above H^{E2 ® F*) = 0. Hence P is a projective bundle over 
JF X 82- From Proposition [STSl , we know that A/K^i, n2, c^i, ^2) C V, and 
since a-semistability is an open condition we have that A/^ni, n2, c^i, ^2) is 
irreducible. □ 

Remark 8.8. If rii < n2, we have an analogous theorem for A/'£(^i, n2, di, ^2) 
via the isomorphism 

Maini, n2, di, d2) = A/'^(n2, rii, -d2, -di) 

given by duality. 



8.3 Moduli space for 2g — 2 ^ a < aM 

Theorem 8.9. Let a be any value in the range 2g — 2 ^ a < om- Then Af^ 
is birationally equivalent to Ml. In particular it is non-empty and irreducible. 

Proof. This follows from Corollary |6.20| and Theorem ^.7| . □ 
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Corollary 8.10. Let {ni,n2,di,d2) be such that GCD{n2,ni + n2,di + d2) = 
1. If a is a generic value satisfying 2g — 2 ^ a < um, then Ma is birationally 
equivalent to Ml, and in particular it is irreducible. 



Proof. From (4) in Proposition |5.4| one has that Ma - 
n2,di + 6/2) = 1 and a is generic. In particular, Ml 



result follows from Theorem B.9 



M^ if GCD(n2,ni + 
= Ml, and hence the 
□ 



9 Moduli space of triples with ni = n2 

Throughout this section we will assume that ni = n2 = n and di ^ d2- 
9.1 Moduli space for di = d2 

Proposition 9.1. Suppose that ui = n2 = n and di = d2 = d. Let T = 

(Ei,E2,4>) be a triple of type (ni, n2, (ii, ^2), and let a > 0. Then T is a- 
(poly)stable if and only if Ei and E2 are (poly)stable and is an isomorphism. 

Proof. In this case = (^m = and hence for every a-semistable triple T = 
[Ei,E2,4>) with a > 0, (f) must be injective and therefore an isomorphism. 
The polystability of Ei and E2 is now straightforward to see. To show the 
converse, suppose that Ei and E2 are both polystable and let T' = {E[, E2, cp') 
be any subtriple of T. 



^la{T') = ixiE[ © E'^) + a- 



2 



n'2 



2 



since n[ ^ for is injective. 



□ 



Corollary 9.2. There is a surjective map from Ma{n,n,d,d) to M{n,d) 
which defines an isomorphism between the two moduli spaces. In particu- 
lar Main, n,d,d) is non-empty and irreducible. 
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Proof. From Proposition 



|0 



it is clear that we have a surjective map, say 



TT : Main, n, d, d) — * M (ra, d) . 

Suppose that vr([T]) = 7r([T']), where [T] and [T'] are points in Afa{n, n, d, d) 
represented by triples T = {E, E, (p) and T' = {E', E', cj)') respectively. We 
may assume that T and T' are polystable triples, and hence that E and E' are 
polystable bundles. Thus, since 7r([T]) = 7r([T']), we can find an isomorphism 
hi : E \—>- E'. Set h2 = <f)' o hi o (p^^ (remember that (p and 0' are bundle 
isomorphisms!). Then [hi, h2) defines an isomorphism form T to T' . Thus vr 
is injective. □ 



9.2 Bounds on Ei and E2 for a > 



Lemma 9.3. Let {Ei,E2,4>) be a triple with ker0 = 0. Let {E[, E2,(f)') be a 
subtriple with n'l = n'2 = n' . Thus we get the following diagram, in which S 
and S' are torsion sheaves: 











Eo 



E' 



El 



E'l 



S 



S' 







Then 



= (Mi?0-/^i)- 

Here s and s' are the degrees of S and S' respectively 

Proof. From the above diagram we get 

n/i2 + s = njj,i , 
nn{E2) + s' = nfi{E'i) . 



— > 0. 

1 /s^ 

2 [y 

-(- 

2\n' 



Thus 



1 (y 1 

/i.(T') = -ifiiE'i) + f,{E'2)) + - = - i2f,{E'2) + - 



a 

+ 2 



'2 (^^(^i) - 9 



a 
2' 



+ -, 



and similarly for HaiT). 



□ 
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Proposition 9.4. Let {Ei,E2,(p) be an a-semistable triple with ker0 = 0. 
Then 



(1) For any subsheaf E[ C Ei 



fx{E[) ^ /ii + ^{n - l)(/ii - /i2) • 



(2) For any subsheaf E2 C E2 



Proof. Since ker = the results of Lemma |9.3| apply. Furthermore, any 
subsheaf E[ C Ei is part of a subtriple {E[, E2,(f)') with n'^ = rig = n'. 
Likewise, given any subsheaf E2 C E2, we can take E[ = 4'{E'2). Thus 
we can use the results of Lemma |9.3| , plus the fact that a-stability implies 
Aq,(T') < for all subtriples, to conclude 



2 \n' n 

for all E[ G El. Similarly 



for all E2 G E2. The results now follow using the fact that ^ s' ^ s and 
1 < n' < n. □ 



9.3 Stabilization of moduli 



Theorem 9.5 (Stabilization Theorem). Let be as in ( \7.1^ ) 

(1) If ai, a2 be any real numbers such that ao < ai ^ ^2, 

J\fa^{n,n,di,d2) ^ Ma2{n,n,di,d2). 

(2) There is a real number ai ^ «o such that 

J\fa^{n,n,di,d2) = J\fa2in,n, di, d2) 
for all ai ^ a2 > a^. 
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Proof. (1). Recall from [7 .31 that if a > ao then any triple, say T = (Ei, E2, (f)), 
in A/'a(n, ra, (ii, 6/2) has rk(0) = n. It follows that in any subtriple, say T' = 
{E[, E2, (j)'), the rank of E[ is at least as big as the rank of E2, i.e. n'l ^ n'g. 
We treat the cases n[ > n'2 and n[ = n'2 separately. In both cases we must 
show that 

if «! ^0:2. If n\ = n'2 then for any a 

K{T') = fi{E[ © E'2) - fi{E, © E2) . (9.1) 

In particular, Aq,(T') is independent of a and hence Aq,j(T') = Aa^iT'). If 
n'^ > 722, then for any a 

A,(T') = fiiE[ © i?^) - /i(Ei © E2) + f - I;) «• (9.2) 

For each subtriple, Aq(T') is thus a linear function of a, with slope 

, / 122 ^\ ^2 ~ ""-'i 

= ^^T^-2j = 2(n; + n'2) ^ ' 

and constant term 

M{T') = fi{E'^ © E'2) - fi{Ei © E2) . (9.4) 

We see that if n'l > n'2 then A(T') < 0. It follows from this that 

A^,{T')^0 =^ A,,(T') ^0 

if «! ^ a2- (2). Consider any ai,a2 such that < «i ^ «2- By Part (1), 
the difference (if any) between A/'aj and AC^j is due entirely to triples which 
are a2-stable but not ai-stable. Any such triple must have a subobject, say 
T' = {E'^,E'2,(p'), for which 

A,,(T') ^0< A,,(T') . (9.5) 

As in (1), we need only consider subobjects for which the rank of E'^ is at 
least as big as the rank of E'2, i.e. n'^ ^ n'2. Clearly ( |9.5| ) is not possible for 
a subobject with n'^ = n'2 (since in that case Aa^(T') = Aa^iT')). Suppose 
then that n'^ > n'2. By ( |9.2| ) and the fact that for such a subobject A(T') < 0, 
we get that 

A.(r) ^ ^ « ^ . (9.6) 
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We claim that there is a bound, ct^, depending only on the degrees and ranks 
of El and E2, such that 

< (9.7) 

for all possible subtriples with n'^ > n'2. For a triple T = {Ei, E2, 4>) in 
Proposition |9]^ applies, giving upper bounds on slopes of subsheaves of both 
El and E2. Using these bounds we compute 



Combined with (p.3|), this gives 



We can thus take 



M(T') nn'i 

^ n{n - l){ni - H2). 



aL = n{n - l){fii - fi2) . (9.9) 



We can now complete the proof of Part (2): if ai > then no triple in Afa2 
can have a subobject satisfying ( |9.6| ). Hence = Mai- □ 



Remark 9.6. If n = 2 then ul = olq = di — d2, i.e. the stabilization parameter 
coincides with the injectivity parameter. 



It is clear from (|9.9| ) that = correspond to the following especial cases. 

Proposition 9.7. The condition a/, = holds if and only if n = 1 or a.^ = 

0. Hence if e is any positive real number: 

(1) If n = 1, the moduli space for every a G {am, 00) is isomorphic to 
A/'a„+,(l, l,di,d2) , 

(2) If am = 0, the moduli space for every a G (0, 00) is isomorphic to 
Me{n,n,di,d2). 
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9.4 Moduli for large a and OL^2g — 2 

Let a > aQ. By Corollary [7731 , we know that all triples in Afa{n, n, di, ^2) are 
of the form 

— >E2^Ei — >S — >0, (9.10) 
where S* is a torsion sheaf of degree d = di — 

Theorem 9.8 (Markman-Xia ||26| ] ) . There is an irreducible family S pa- 
rameterizing quotients Ei — > S — > 0, where Ei is a rank n and degree di 
locally free coherent sheaf varying on a hounded family, and S is a torsion 
sheaf of degree d > 0. 



Theorem 9.9. Let a > ao, then N'a{n,n,di,d2) is irreducible. 



Proof. Since a > aQ, an a-semistable triple T = {Ei,E2,4>) defines a se- 



quence as in (9.10) and hence a quotient Ei — > S — > in S. That Ei 



varies on a bounded family is a consequence of (1) in Proposition |9.4| . In- 
deed, let E he a. vector bundle of degree d and rank n satisfying 

/i(E') ^ B (9.11) 

for all subbundles E' C E, and fixed B. But then (see, for instance, the 
proof of Theorem 5.6.1 in ||2^) we can find a line bundle L of sufficiently 
high degree such that H^{E ^ L) = for all E which satisfy ( p.ll|) . The 
irreducibility of Ma{n, n, di, ^2) follows now from the irreducibility of S and 
the fact that a-semistability is an open condition. □ 



By analogy with the ni 7^ n2, let us denote by Mhin, n, di, ^2) the 'large 
a' moduli space, i.e. the moduli space of a-semistable triples for any a e 
(aL,oo). Since ^ ao we have that all triples in ML{n,di,d2) are of the 
form 

— > E2^ El — >S — >0, 



and that Ei and E2 are bounded by the constraints in Proposition P^ . 
In the converse direction we have 

Proposition 9.10. Let T = {Ei,E2,4>) be a triple such that kei (p = If 
El and E2 are semistable, then T is a-semistable for large enough a, i.e. 
T G Af{n, n, di, 6/2). // either Ei or E2 is stable, then T is a-stable. 
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Proof. Since ker = 0, it follows (as in the proof of Theorem p.5| ) that in 
any subtriple, say T' = {E[, E2,(j)'), the rank of E[ is at least as big as the 
rank of E'^, i.e. n[ ^ n'^. If n[ > n'^, then M, (|0) and M apply, with 



A(T') < and 3^^^ ^ a^. Thus < whenever a > ai- For subtriples 
with n[ = n'2, equation (|9.1|) apphes, i.e. 

A„(r') = fi{E[ © E'^) - ^i{E^ © E2) 

for any a. For such subtriples, and for any a, it thus follows that 

(1) Aa(T') ^ if both El and E2 are semistable, and 

(2) Aq,(T') < if at least one of the bundles is stable. 



□ 



Theorem 9.11. The moduli space J^l{n,n,di,d2) is non-empty. 

Proof. Our strategy is to show that there exist rank n stable bundles Ei and 
E2 of degree di and ^2, respectively, and a torsion sheaf S of degree di — d2, 
fitting in an exact sequence 

— > E2 — > El — ^ S — ^ 0. 

The result will then follow from Proposition p. 10 . 

To prove this, let E he a vector bundle, and let Quot'^(i?) be the Quot 
scheme of quotients E — > S where 5* is a torsion sheaf of degree d. The 
basic fact we will need is the following result of R. Hernandez (|^): Let 
ijj : — > S be an element in Quot'^((9"), then for a generic S, the vector 
bundle E = keitp is stable. Notice that degE = —d. By tensoring with a 
line bundle L of big enough degree (depending on n and d), we can extend 
the result to Quot'^(L"'), so that the kernel of any element in Quot'^(L") has 
a given (fixed) degree. 

Let L be a line bundle of degree m and d" > such that di = nm — d". By 
Hernandez resuh, if ^ : L" — ^ S" G Quot"'"(L") is generic, then Ei = ker ijj 
is a stable bundle of rank n and degree di. Let d = di — d2 and consider 
a generic element rj : Ei — > S G Quot'^(i?i). Let E2 = ker 77, and let S' 
the cokernel of the natural inclusion E2 — > L^. We have the following 
commutative diagram: 
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II i i 

^ E2 ^ L"" ^ S' ^ 

S" ^ S" 

i i 
0. 

Wc sec from the diagram that E2 coincides with the kernel of L" — > S'. 
If 5" is general enough we can again apply the basic result of Hernandez and 
conclude that E2 is stable. To show that this is indeed the case, we observe 
that the diagram defines a map 

Quot^(Ei) X Quotf (L") QuoC"{L^), 

where Quotg denotes an open non-empty subscheme of Quot, which is sur- 
jective and finite. □ 

Proposition 9.12. The moduli space N'L{n,n,di,d2) is birationally equiva- 
lent to a F^-fibration V over M'{n, 4) X Diy'^{X), where N = n(o?i -^2) - 1. 

Proof. Let E2 be a rank n and degree 0^2 vector bundle and let S he a, torsion 
sheaf of degree d > 0. We construct Ei as an extension 

— >E2 — >Ei — >S — >0. (9.12) 

Such extensions are parameterized by Ext ^(5, -£'2)- Suppose that S is of the 
form S = Od, where D is a divisor in Div'^(X). Let L be a line bundle. 
Consider the short exact sequence 

— > L*{-D) — >L* — >Od — ^ 0, 

and apply to it the functor Hom(-, E2), to obtain the long exact sequence 

H\E2®L) H\E2®L{D)) 

Ext^(OD,^2) — > H\E2®L) — > H\E2®L{D)) — > 0. ^ ' 

We thus have 

dim Ext^ (0^5,^2) = x{E2 ®L)- x{E2 ® L{D)) = nd, 
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which is positive since we are assuming d > 0. Taking L so that deg(L) >> 0, 
we have that H^{E2®L) = 0. If E2 is semistable (or more generally it moves 
in a bounded family) we can take the same L for every E2. Then 

Ext^(C^, E2) = H\E2 ® L{D))/H\E2 ® L). 



Let V be the set of equivalence classes of extensions (|9.12 ), where E2 is stable 
then "P is a P^-fibration over M'{n,d2) X Div'^(X), where N = nd - 1 = 
dimF{Ext^{OD, E2)). Setting ci = rfi — ^2, a simple computation shows that 

dimP = {g — + — nin2) — ^1^2 + ?t-2'^i + 1- 

Cleary V is irreducible of the same dimension as Ml-, and since it is con- 
tained in S (like Ml) it must birationally equivalent to Ml- Notice that If 
GCD(n, (^2) = 1, then P is a Picard sheaf. □ 



Combining Theorems |9.9| and |9.11| , and Proposition |9.12| we have the 
following. 

Theorem 9.13. . The moduli space AfL{n,n,di,d2) is non-empty and irre- 
ducible. Moreover, it is birationally equivalent to a -fibration over M^{n., ^2) x 
Div'^(X), where the fiber dimension is N = n{di — ^2) — 1- 



Theorem 9.14. Let a^2g-2. Then 

(1) The moduli space is birationally equivalent to Ml and it is hence 
non-empty and irreducible. 

(2) // in addition either 

• GCD(n, 2n, di + ^2) = 1 and a ^ 2g — 2 is generic, or 

• di — d2 < a , 

then N'a{n,n,di,d2) is birationally equivalent to A/L(n, n, rfi, ^2) and 
hence irreducible. 



Proof. (1) From Theorem ^]9| we know that Ail is birationally equivalent to 
A/"/. The result follows now from Corollary |6.20| and Theorem |9.13| . 



(2) For the first part, we observe that from (4) in Proposition one 
has that Afa = if GCD(?t,, 2n, di + ^2) = 1 and a is generic, and hence 
the result follows from (1). The second part is a consequence of Theorem 



9.9. 



□ 



89 



10 Existence and connectedness for U(p, q) and 
PU(]9, q) moduli spaces 

We now return to the representation spaces 7l(P\J{p,q)) and 7lr(JJ{p,q)), 
defined in section 0. Recall that we identified components of 7l(PU{p,q)) 
labeled by [a,b] G Z © Z/(p + g)Z, and similarly identified components of 
^r(U(p, q)) labeled by (a, 6) G Z © Z. 

By Proposition U 7^^(a,6) is a U(l)29-fibration over TZ[a,b], and hence 
the number of connected components of 7?.r(a, b) is greater than or equal to 
that of TZ[a,b]. By Proposition |3.12| there is an homeomorphism between 



7^r(a, b) and the moduli space Ai{a, b) of U(p, g)-Higgs bundles and this re- 
stricts to give a homeomorphism between the subspace 7^p(a, b) of irreducible 
elements in TZr{a,b) and the subspace A4^{a,b) of stable Higgs bundles in 
A4{a, b). By Proposition |4]^ the number of connected components of A4{a, b) 
is determined by the number of connected components in the subspace of lo- 



cal minima for the Bott-Morse function defined in Section CT. By Theorem 
|5.9| we can identify the subspace of local minima as a moduli space of a-stable 
triples, with a = 2g — 2. Summarizing, we have: 



|7^o(7^[a,6])| ^ |7^o(7^^(a,&))| [Proposition |2]2 



^ |7ro(A^(a,6))| [Proposition |Xg 



^ I TTo (A/'(a, b) ) I [Proposition |42 



ko(A/'2g_2('n'i, ^^2, di,d2))\ [Proposition 



where |vro(-)| denotes the number of components, and (in the notation of 
Section ^ the moduli space of triples which appears in the last line is either 
U2g-2{p,q,a + p{2g-2),b) (ifa/p^b/q) or Af2g-2{q,P,b + q{2g - 2),a) (if 
a/p ^ b/q). Similarly, replacing Proposition |4.2| with Proposition [4.18| , we 
get that 

\Tio{n*[a,b])\ ^ \noif/2g-2i^i^^2,di,d2))\ . 

In particular, if the moduli spaces of triples are connected, then so are the 
spaces 7l[a, b] and '}Z*[a, b]. 

We use the results of Sections 1^-^ to determine the connectedness of the 
spaces A/'23_2(p, g, a + p{2g - 2), 6) and Af2g-2{q,P,b + q{2g - 2), a). Our 
main tool is Corollary |6.2CI| , which allows us convert the problem into one 
of counting the components for A/l, the moduli space of a-stable triples for 
sufficiently large a. In the case p ^ q^ Theorem F?^ and its dual supply the 
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requisite details. The results in Section ^ cover the the case p = q, which is 
not as well understood as the case p ^ q. 

Before presenting specific results, we make some general observations. 

10.1 Toledo invariant and the number of components 

As described in Section H.4L it is useful to introduce the combination 



2 

r := r(a, b) = —{qa — pb) 
n 

known as the Toledo invariant. Throughout this section, for convenience, we 
set n = p + q. 

The map (a, b) i— > r(a, b) defines an invariant on each component 7?.r(a, b). 
Moreover, since the map factors through Z©Z/ {p+q)'Z, it defines an invariant 
of TZ[a, b], where it takes the same value as on TZr{a, b). 

Proposition 10.1. Suppose that GCD(p, g) = k. Then the map 

2 

T :Z©Z/(p,g)Z — > -Z 

n 

2 

[a, b] —{aq — bp) 
n 

fits in an exact sequence 

> Z/kZ Z©Z/(p,g)Z fZ > (10.1) 

where the map a is [t] [t|,t|]. In particular, r is a k : 1 map onto the 
subset ^Z C -Z. 

n n 

Proof. The map a is clearly injective, and r o a = 0. To see that ker(r) = 
im((T), observe that if r[a, 6] = then either a = 6 = 0or| = |, i.e. 
[a, b] = [tf , if ] for some t G Z. Finally, if a^q — b^p = k, then for any I G Z 
we have r[/ao,/&o] = Thus r is surjective onto ^Z. □ 

Remark 10.2. Proposition |10.1| shows why we mustQuse [a, b] rather than r 
to label the components of lZ{y\]{jp,q)) or of 7?.r(U(p, g)). There are, 
nevertheless, important features of TZ[a,b] and TZr{a,b) which depend on r 
rather than on [a,b] or (a, 6). The rigidity result described in Section [3l^ is 

'^Unless p and q are coprime, in which case the correspondence between [a, b] and r is 
1-1. 
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one such feature. Others emerge in the analysis of the minimal submanifolds, 
i.e. in the moduli spaces J\f2g~2{p, Q, '2 + p(25' — 2), b) and N'2g-2{q,P, b + q{2g — 
2), a). In particular, for triples of the indicated type, the key critical values of 
the stability parameter a (i.e. a^, ao? Q^t; QiM, in the notation of Section H-^, 
as well as the all-important location of 2(7 — 2 within the interval (a^, oa/ ), 
are all determined by r. 

Proposition |10.1| allows us to count the number of components in the 
decomposition 7^(PU(p, g)) = U(p g) ^f*^' Recall (from Section |3^) that if 
p ^ q then the Toledo invariant is bounded by |r| ^ tm = 2p{g — 1) . Thus 
the set of allowed values for r is contained in [—tm, tj\/] fl |Z. 

Definition 10.3. Suppose that GCD(p, g) = k. Define 

2k 

C = T-\[-ni,rM]n—Z) , (10.2) 



where r is the map defined in Proposition \10.1 . 



The following is then an immediate corollary of Proposition 10.1 



Corollary 10.4. Suppose that GCD(p, g) = k and C is as above. Then C 
is precisely the set of all the points in'LQl'/ {p, q)TL which label components 
TZ[a,b] in 7?.(PU(p, g)). The cardinality of C is 

\C\ = 2nmin{p, q}{g — 1) + k 

= lii-rAuTM] n -Z)| + GCD(p,g) - 1 . 
n 



Proof. The first statement is a direct consequence of Proposition |10.1| and 
the bound on r. Suppose for definiteness that min{p, g} = p. Then since 
Tm = 2mm{p, q}{g — 1) = ^{n^{g — 1)) G the number of points in 

[—Tm, Tm] n is 2n^(g — 1) + 1. The second statement now follows from 
the fact that r is a A; : 1 map. The proof is similar if min{p, g} = g. □ 



10.2 Genericity of 2g-2 and coprimality conditions 

In general the Higgs moduli spaces Ai (a, b) and also the triples moduli spaces 
A/2g-2(p, g, a' + p{2g — 2), 6) and A/2c,-2(g,P, b + q{2g — 2), a), are not smooth. 
In both the Higgs and the triples moduli spaces, the singularities occur at 
points representing strictly semistable objects. Apart from complications 
caused by such singularities, further difficulties can arise if 25f — 2 is not a 
generic value in the range of the stability parameter for the triples. 
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We now show that both types of problems are avoided if we make the 
assumption that p + q and a + h are coprime, i.e. that GCD(p + g, a + 6) = 1. 



Proposition 10.5. Suppose that GCD(p + g, a + 6) = 1. Then 

1. A4{a,b) is smooth, 

2. a = 2g — 2 is not a critical value for triples of type {p,q,a+p{2g — 2),b) 
or {q,p,b + q{2g — 2), a) , and 

3. The moduli spaces Af2g-2iP,<l,ci+p{'^g — '^),b) and Af2g-2i(l,P,b + q{2g — 
2), a) are smooth. 

Proof. The first statement is simply a restatement of Remark p.l3| . The 



second and third statements follow from Lemma f).7\ Indeed, for triples of 



type {p, q,a + p{2g — 2), 6), the coprime condition in that lemma becomes 
GCD(p + q,a + b + {2g - 2){p + q)) = 1 . (10.3) 



But ( |1U.3| ) is equivalent to GCD{p + q,a + b) = 1. The proof is similar for 



triples of type {q,p, b + q{2g — 2), a). □ 

Recall that p and q are fixed, but [a, b] runs over C, the indexing set for 
the components 7l[a, b]. The coprime condition GCD(p + q,a + b) = 1 can 
thus be satisfied on some components but not on others. 

Proposition 10.6. Fixp andq and letC C Z©Z/(p+g)Z be as in Definition 
llO.dj . Let C(i) denote the subset of classes [a, b] E C for which the condition 
GCD(p + q,a + b) = 1 is satisfied. Then both C(i) and its complement in C 
are non-empty. 

Proof. If a = p and b = q — 1 then GCD(p + g, a + 6) = 1. Also, r(p, g — 1) = 
which is in [—tm,tm] H ^Z. Thus [p,q — 1] is in C(i). It is similarly 
straightforward to see that (p, g) = (0,0) defines an element in C — C(i), as 
does (p, g) = {p, —p) if p ^ g or (p, g) = (g, — g) if g ^ p. □ 

It seems somewhat complicated to go beyond this result and completely 
enumerate the elements in C(i). The following result is, however, a step in 
that direction. 

Definition 10.7. Let 1] C M © R be the region bounded by 
• the ray b = q and a ^ p, 
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• the ray a = p and b ^ q, 

• the ray a = and b ^ 0, 

• the ray 6 = and a ^ 0, 

• the line aq — bp = np{g — 1), and 

• the line aq — bp = —np{g — 1), 

and including all the boundary lines except the first two rays. Let Qz be the 
set of integer points in Q, i.e. Qz = ^^H^ ® ^- refer to Qz as the 
fundamental region for {p, q) . 

Proposition 10.8. Suppose that p and q are integers with GCD(j9, q) = k 
and p ^ 00. 

1. There is a bijection between C and Viz- 

2. If (a, b) lies in flz then d = a + b satisfies the bounds 

-n{g-l)^d<n. (10.4) 
All values of d in this range occur. 

3. Let It denote the line aq — bp = tk. Then the points on U H^z define 
the locus of points (a, b) for which T(a, b) = t^. 

4. The line k intersects Qz for —^{g — 1) ^ t ^ ^{g — 1) For each integer 
t in this range, there are k points on lt{^^z- 

5. For a fixed t, all integer points {a, b) & It f]^lz h(^'i^^ the same GCD{d,^), 
where d = a + b. 

6. If GCD(c/, f ) ^ 1 then GCD(rf', n) ^ 1 for all (a', b') G fj 

Proof. (1) Suppose first that ^ ^ ^. Pick / such that ^ a + ^ p. Then 
b + Iq ^ q, so that (a + lp,b + Iq) is in the fundamental region. Similarly, if 
^ ^ ^ then we pick I such that ^ b + lq ^ q and see that a + lp ^ p. In this 
way we get a well defined map from C to the fundamental region. The map 
is clearly injective. To see that it is surjective, notice that the boundary lines 
aq — bp = np{g — 1), and aq — bp = —np{g — 1) correspond to the conditions 
T = tm and r = —tm respectively. 

^With an analogous Proposition for the case q. 
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(2) This is clear from a sketch of the fundamental region. In such a sketch, 
the loci of points with constant value of d = a + b are straight lines of slope 
— 1. Since p ^ q, the extreme cases are those of the lines passing through the 
points (0, —n{g — l) and (p, q). Using the points (0, b) with —n{g — l) ^b < q 
we get points at which all values of d in the range —n{g — 1) ^ d < q are 
realized. We get values of d in the range g ^ d < n at the points (a, q — 1), 
with 1 ^ a < p. 

(3) - (4) This is simply a re-statement of Proposition 10. 1| . 



(5)-(6) Both follow from the fact that for any two points (a, b) and (a', b') 
on It, we get d' = d + for some s E Z. □ 

Remark 10.9. There is no converse to (6), i.e. it is possible to have GCD((i', n) ^ 
1 for some {a',b') G ltf]Qz, even if GCD{d,j) = 1. For example, take 
p = 2, g = 4, a = -1, 6 = 0, a' = 0, 6' = 2, and t = -2. Then GCD(rf', n) = 2 
but GCD(rf, f ) = 1. 



10.3 Moduli spaces of U{p, g)-Higgs bundles 

In this section we give our main results on the moduli spaces of U(p, g)-IIiggs 
bundles. As noted several times before, the results are conveniently stated 
using the Toledo invariant r(a,6) = 2{qa — pb)/{p + q). For completeness 
we recall from Remark |3.23| that the maximal value of the Toledo invariant 
is tm = m.in{p,q}{2g — 2). In the case p = q these definitions simplify to 
T = a — b and Tm = p{2g — 2). 

Recall from Proposition |3.15| that, whenever the moduli space Ai^{a,b) 



of stable U(p, g)-IIiggs bundles with invariants (a, b) is non-empty, it is a 
smooth complex manifold of dimension 1 + {p + q)'^{g — 1). We shall refer to 
this dimension as the expected dimension in the following. 

Theorem 10.10. Let {a,b) be such that T{a,b) = 0, then A4{a,b) is non- 
empty and connected. 



Proof. It follows from (1) in Proposition |5.12| , applied to the triples in J\f{a, b) 



that if r = then | = ^- The result is thus simply a re-statement of (3) in 
Theorems |5.11| and ^]9| (plus the non-emptiness of the moduli spaces M(p, a) 
andM(g,6)). □ 

Remark 10.11. It is not clear if M.^{a, b) is non-empty. Notice that the min- 
imal subvariety is Af{a,b) = M{p,a) x M{q,b), in which every element is 
reducible. To show that strictly stable points exist we must look beyond 
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the minima. We observe though, that if it is non-empty, then Ai'^{a,b) is a 
smooth connected manifold of the expected dimension. 



Theorem 10.12. Let {a,b) be such that < |r(a,6)| < tm- Then the clo- 
sure A4^{a,b) of the moduli space of stable \J{p,q)-Higgs bundles with fixed 
invariant {a,b) is connected. Moreover, the stable locus Ai^{a,b) is a smooth 
non-empty manifold of the expected dimension. 



Proof. \i p ^ q, from (2) of Proposition |5.12| we have that 2g — 2 < au and 
therefore Theorem |8.9| shows that ■f^2g-2 is non-empty and irreducible. If 



p = q, Theorem ^.14| shows that ■f^2g~2 is non-empty and irreducible. From 



this we draw two conclusions. Firstly, we see from (1) and (2) of Theorem ^.11 



that M.^{a,b) is connected. Secondly, it follows that M.^{a,b) is a smooth 
non-empty manifold of the expected dimension. □ 

Theorem 10.13. Let p = q and let {a,b) be such that |r(a,6)| = tm- Then 
A4{a,b) is non-empty and connected. Moreover, Ai''^{a,b) is non-empty, and 
smooth of the expected dimension. 

Proof. Since p = q, the Toledo invariant is r = a — b. Suppose for definiteness 
that a > b. Since, by hypothesis, a — b = tm = p{2g — 2), the moduli space 
of triples J\f2g-2{p,P,b + p{2g — 2), a) is simply A/2£,_2(p, P, a, a), which by 
Corollary |9.2| is irreducible. Now, by (2) in Theorem |5.11| , we have that 



A^(a,6) is connected. A similar argument applies if a < 6. □ 



If p 7^ g then Theorem |3.26| gives a description of Ai (a, b) when (a, b) is 
such that |r(a,6)| = Tm- For definiteness, assume that p < q and T{a,b) = 
Tm- Then, using the more precise notation J^{p, q, a, b) for the moduli space 
of semistable U(p, g)-Higgs bundles with topological invariant (a, 6), Propo- 
sition p.26| gives 



Mip,q,a,b)=Mip,p,a,a-p{2g-2)) x M{q - p,b - a + p{2g - 2)), 

where M{q — p,b — a + p{2g — 2)) is the moduli space of polystable 
bundles of degree q — p and rank 6 — a + p{2g — 2). 

The dimension at a smooth point in /iA{p,q,a,b) is 2 + (p^ -|- 5q^ — 
2pq){g — l), and it is hence strictly smaller than the expected dimension. 

Every element in Ai{p,q,a,b) is strictly semistable, i.e. A^*(a, 6) is 
empty. 
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Similar statements hold for p > q and/or r = —tm- We can now add: 

Theorem 10.14. Let p ^ q and let {a,b) be such that |r(a,6)| = tm- Then 
A4{a,b) is non-empty and connected. 



Proof. This follows from (1) and (2) of Theorem [5.11| and Corollary |8.2| . □ 



Theorem 10.15. Let {a,b) be such that \r{a,b)\ ^ tm, and suppose more- 
over that GCD(p + g, a + 6) = 1. Then M.{a, b) is a non-empty, connected, 
smooth manifold of the expected dimension. 

Proof. Under the condition GCD(p + g, a + 6) = 1 there are no strictly 
semi-stable U(p, g)-Higgs bundles and hence M.{a,by = M.{a,b) (cf. Re- 
mark |3.13|) . 



Note also that the condition GCD(p-|-g, a-\-b) = 1 excludes the possibility 
r = and, when p q, the possibility |r| = tm (by Proposition |3.26|) . Hence 
we have, in fact, that < |r| ^ tm and, when p q, that < |r| < tm- 

Thus the statement is immediate from the preceding Theorems. □ 

In the case p = q and {p — l){2g — 2) < |r| ^ tm we do not need to 
impose any coprimality conditions in order to prove connectedness of the 
moduli spaces Ai{a,b). This follows from the fact that in this situation no 
flips are required to go from A/l to Af2g-2- Thus we have the following. 

Theorem 10.16. Assume that p = q and that r = a — b is such that 

{p-l)(2g-2)<\r\^rM=pi2g-2), 

then /iA{a,b) is non-empty and connected. 

Proof. For definiteness let us assume that a > b (an analogous argument 
applies to a < b). We must then prove the connectedness of the moduli 
space of {2g — 2-polystable) triples Af2g-.2{p,P, o, + p(2g — 2),b) Recall from 



([715| ) that 

ao = di — d2 = a — b -\- p{2g — 2). 

Since, by hypothesis, a — b > {p — 1)(2(7 — 2) we have that 2g — 2 > ao. We 
thus have that Af2g-2{p,P, d + p{2g — 2),b) is non-empty and irreducible by 
Theorem |9.14|, and hence A4{a, b) is connected by Theorem |5.11|. □ 



Remark 10.17. The connectedness part of Theorem |10.16| is a result previ- 
ously proved by Markman and Xia . 
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10.4 Moduli spaces of U(]7, and PU(]7, representa- 
tions 



In this section we finally give our results on the moduli spaces of representa- 
tions. We begin by translating the results of Section |10.3| into results about 



the moduh spaces of U(p, q) representations via Proposition |3.12| . We denote 
by 1Z^{a, h) the closure of 7?.f (a, h) in 7?.r(a, h). 

Theorem 10.18. Let {a,b) be such that |r(a,6)| ^ tm- Unless further re- 
strictions are imposed, let (p, q) be any pair of positive integers. 



(1) // either of the following sets of conditions apply, then the moduli 
space TZ^{a,b) of irreducible semi-simple representations, is a non- 
empty, smooth manifold of the expected dimension, with connected clo- 
sure TZ^i^a, b): 

{i) < |r(a,6)| < tm , 
(ii) |'r(a, b)\ = tm and p = q 

(2) // any one of the following sets of conditions apply, then the moduli 
space TZri^, b) of all semi-simple representations is non-empty and con- 
nected: 

{i) r(a,6) = 0, 

{ii) |r(a, b)\ = Tm and p ^ q. , 

(m) {p - l){2g - 2) < |r| ^ tm = p{2g - 2) and p = q, 

(tv) GCD(p + g, a + 6) = 1 

(3) // |r(a, 6)1 = tm and p ^ q then any representation in 7lr{a,b) is 
reducible (i.e. 7^f(a, 6) is empty). If p < q, then any such representa- 
tion decomposes as a direct sum of a semisimple representation ofV in 
U(p, p) with maximal Toledo invariant and a semisimple representation 
in U(g — p). Thus, if r = p{2g — 2) then there is an isomorphism 

'JZr{p,q,a,b)='}Zr{p,p,a,a-p{2g-2)) x R^(q - p^b - a + p{2g - 2)), 

where the notation 7lr{p, q, a, b) indicates the moduli space of represen- 
tations ofV in U(p, g) with invariants {a,b), and Rr{n,d) denotes the 
moduli space of degree d representations ofV in U(?7,). (A similar result 
holds if p > q, as well as if r = —p{2g — 2) ). 
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(4) // GCD(p + q,a + b) = 1 then 7lr{a,b) is a smooth manifold of the 
expected dimension. 



From Theorem p.0.18| and Proposition we finally obtain the following 
theorem about the moduli spaces of PU(p, q) representations. Note that 
the closure 'R*[a,b] in TZ[a,b] is the image of TZ^ia^b) under the map of 



Proposition p.2| , hence these two spaces have the same number of connected 
components. 

Theorem 10.19. Let {a,b) be such that |r(a,6)| ^ tm- Unless further re- 
strictions are imposed, let {p, q) be any pair of positive integers. 



(1) // either of the following sets of conditions apply, then the moduli space 
7i*[a,b] of irreducible semi-simple representations, is non-empty, with 
connected closure TZ* [a, b] : 

{i) < |r(a,6)| < tm , 
(a) |r(a, 6)1 = Tm and p = q 

(2) // any one of the following sets of conditions apply, then the moduli 
space TZ[a,b] of all semi-simple representations is non-empty and con- 
nected: 

(t) r(a,6) = 0, 

(ii) |r(a, b)\ = Tm and p ^ q. , 

(m) {p - l){2g - 2) < |r| ^ tm = p{2g - 2) and p = q, 

(tv) GCD(p + q,a + b) = l 

(3) // |r(a,6)| = r^j and p ^ q then any representation in TZ[a,b] is re- 
ducible (i.e. TZ*[a,b] is empty). If p < q, then any such representation 
reduces to a semisimple representation ofrciX in P(U(p,p) x U(g— p)), 
such that the representation in PU(p, p) induced via projection on the 
first factor has maximal Toledo invariant. (A similar result holds if 
p > q, as well as if r = —p{2g — 2) ). 



□ 



Remark 10.20. As explained by Hitchin in [^3], Section 5], the moduli space 
of irreducible representations in the adjoint form of a Lie group is liable to 
acquire singularities, because of the existence of stable vector bundles which 
are fixed under the action of tensoring by a finite order linebundle. For this 
reason we are not making any smoothness statements in Theorem |10.19| . 
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